Teaching an AI Agent to Say \\\"I Don't Know\\

Language models are trained to be helpful. They want to give you an answer. When they don't have an answer, they'll often make one up and present it with the same confidence as a real fact. In a trivia game, that's mildly amusing. In a travel booking system where the AI handles real money, it's dangerous.

A hallucinated flight number means the user shows up at the airport with a booking that doesn't exist. A hallucinated price means the charge doesn't match what was promised. A hallucinated travel requirement means the user arrives at a border without the right visa.

We've spent significant engineering effort teaching our AI agent when to say "I don't know" or "Let me check that" instead of inventing an answer. Here's why that's harder than it sounds.

Uncertainty vs. hallucination

These are different things, and the distinction matters for how you build guardrails.

Uncertainty is when the model doesn't have enough information and knows it. "I'm not sure what your budget is. What range are you thinking?" That's healthy. The model is calibrated about its own knowledge gaps.

Hallucination is when the model doesn't have information but generates something plausible anyway, without flagging that it's speculating. "Flight AA2847 departs at 9:15 AM and costs $523" when no such flight exists. The model is confidently wrong.

The technical challenge is that language models don't have a built-in uncertainty meter. They don't know what they don't know. Every output feels equally "real" to the model, whether it's recalling a fact from training data or improvising on the spot.

Our guardrails work at multiple levels to catch both problems.

Guardrails for financial transactions

The highest-stakes hallucination domain is pricing and booking. A wrong restaurant recommendation is a bad evening. A wrong flight booking is hundreds or thousands of dollars.

We enforce strict guardrails around any operation that involves money.

Real-time verification. The agent never quotes a price from memory or from a previous search. Every price shown to the user comes from a live API call with a timestamp. If the user asks "how much was that flight?" about a search from an hour ago, the agent re-checks the price rather than repeating the cached number. Prices change fast in travel.

Explicit confirmation gates. Before any payment, the agent presents the exact booking details (flight number, dates, price, passengers) and requires the user to confirm. This catches cases where the agent misinterpreted which option the user selected.

No autonomous spending. The agent can search and recommend, but it cannot complete a booking without the user explicitly saying yes to the specific transaction. We maintain a full audit trail for every AI-initiated action. No amount of conversational momentum ("sounds good, go ahead") bypasses the confirmation step for the actual payment.

Anomaly detection. If a booking price is significantly different from what was discussed in conversation, the agent flags it. "The price I'm seeing now is $847, which is $50 more than when we searched earlier. Want to proceed or should I look again?"

Detecting low-confidence responses

Since the model doesn't natively report confidence, we infer it from signals.

Hedging language. When the model generates phrases like "I think," "probably," "it's possible that," those are weak confidence signals. We detect these patterns and, in contexts where accuracy matters (pricing, schedules, travel requirements), route them to verification rather than letting the hedged answer through.

No-source claims. When the agent makes a factual claim that didn't come from a tool call (no flight search was run, no hotel database was queried, no reference data was checked), that's a hallucination risk. We flag responses that contain travel-specific claims without corresponding tool results.

Inconsistency detection. If the agent's response contradicts information from earlier in the conversation or from a tool result, something is wrong. We check for basic consistency and flag mismatches.

When any of these signals fire, the agent's behavior changes. Instead of presenting the low-confidence information as fact, it says something like "I want to make sure I give you accurate info. Let me check that." Then it runs the appropriate tool to get verified data.

Building trust through honest limitations

Here's a counterintuitive product insight: users trust an AI MORE when it admits uncertainty than when it confidently generates plausible-sounding answers.

"I'm not sure if Japan requires a visa for your nationality. Let me look that up." followed by a verified answer builds trust. The user sees that the agent is checking its work. They know the answer is verified, not guessed.

Compare that to an agent that confidently states "You won't need a visa for Japan" based on pattern matching from training data, without checking the user's nationality or current requirements. Even if it's right this time, the confidence is unearned. And the one time it's wrong, the user has a serious problem.

70% of travelers say they're open to AI-assisted trip planning. Trust is the gating factor. And trust is built in the moments where the AI could have guessed but chose to verify instead.

How competitors handle AI errors

Most major travel companies that have shipped AI features take one of two approaches when the AI gets something wrong.

The first is hiding the AI. The chatbot generates a response, and if it seems off, the system quietly routes the user to the traditional interface. "Let me take you to our search page for that." The AI disappears. This avoids the hallucination problem by avoiding the AI entirely whenever it's uncertain.

The second is disclaimers. "AI-generated responses may contain errors. Please verify all information." This is the corporate equivalent of "not my problem." It puts the verification burden on the user, which defeats the purpose of having an AI assistant.

We take a third approach: the agent stays in the conversation, acknowledges what it doesn't know, checks when it needs to, and provides verified information. It doesn't disappear and it doesn't disclaim. It works harder.

This requires more engineering. Confidence detection, verification pipelines, graceful conversational recovery. But it creates an AI that users actually trust to handle important tasks, which is the whole point.

AI hallucination is especially dangerous when financial transactions are involved. The only acceptable response to uncertainty in a booking system is verification, not fabrication. We'd rather the agent take an extra second to check than give an instant wrong answer. Human confirmation is required for all payments, and every intermediate fact that leads to that payment should be verified too.

The goal isn't an AI that's always right. That's not achievable with current technology. The goal is an AI that's always honest about what it knows and what it needs to check. That's achievable with the right engineering, and it's the foundation for the kind of trust that makes AI-powered booking work.

Nowah is an AI travel agent that searches and books real flights and hotels through conversation — no filters, no thirty open tabs. Plan your next trip.