Designing for AI Uncertainty: When the Agent Isn't Sure

Here is a truth about AI products that most companies do not want to talk about: the AI is not always right. It does not always understand what the user wants. It does not always find what the user is looking for. And sometimes, it simply does not know the answer.

Most AI products handle this by pretending it does not happen. The AI responds confidently regardless of how certain it actually is. It generates plausible-sounding text whether or not that text is accurate. It presents results without indicating whether they are exactly what the user asked for or a best-guess approximation.

This is a problem. Over-confident AI is worse for trust than uncertain AI. When a user discovers that the AI was wrong but sounded sure, they stop trusting everything it says. When a user sees the AI acknowledge its limits honestly, they trust the things it is confident about.

We designed Nowah to communicate uncertainty explicitly. This is harder than it sounds, and we believe it is one of the most important design challenges in AI products.

The spectrum of AI certainty

AI responses do not fall neatly into "right" and "wrong." There is a spectrum.

On one end: high confidence. The user asks "What time does my flight depart?" The agent has the booking data. The answer is factual and specific. "Your United flight UA 874 departs at 8:15 AM from Terminal 7." No uncertainty. No hedging needed.

In the middle: moderate confidence. The user asks "What is a good neighborhood to stay in Lisbon?" The agent has information about Lisbon neighborhoods, but "good" is subjective and depends on what the user cares about. The answer should be helpful but framed as a recommendation with reasoning, not a fact. "Alfama is popular for its historic charm and views. If you want nightlife, Bairro Alto is the spot. For a quieter stay, Principe Real has great restaurants and parks."

On the other end: low confidence or inability. The user asks "Can I bring my pet tortoise on a Ryanair flight?" The agent may not have specific data about Ryanair's exotic pet policy. Making up an answer risks giving dangerous misinformation. The right response: "I'm not certain about Ryanair's policy on exotic pets. I'd recommend checking directly with Ryanair or calling their customer service for this one."

Most AI products treat all three scenarios identically. The text sounds the same level of confident regardless of certainty. The user has no way to distinguish "the AI knows this for a fact" from "the AI is making an educated guess" from "the AI has no idea but generated something plausible."

We use different response patterns for different certainty levels, and we believe users can feel the difference even when they cannot articulate it.

Clarification without interrogation

When the AI does not have enough information to give a useful answer, it needs to ask follow-up questions. This is where many chatbots go wrong. They interrogate.

"Where do you want to go? When do you want to go? How many travelers? What class? What is your budget? Do you prefer direct flights? Any airline preference?" Seven questions before a single useful response. The user feels like they are filling out a form verbally. The conversation feels like a data collection exercise, not a dialogue.

Good clarification asks one question at a time and asks only what is needed. If the user says "I want to go to Japan," the agent does not need to know everything immediately. It can start by asking about dates, which is the one piece of information required to search for flights. Preferences, airline, class, budget, these can come from memory (if the user has used the app before) or be applied later.

The phrasing of clarification questions affects how they feel. "Please specify your preferred travel dates" is robotic. "When are you thinking? I can be flexible if you are" is conversational. "What dates work for you?" is direct without being cold. We write clarification questions in the same voice the agent uses for everything else: natural, direct, helpful.

There is a specific pattern we call "progressive narrowing." The agent starts with the most open version of the question and narrows based on responses. "When are you thinking?" might get "sometime in April" which narrows to "Any particular week, or should I check the whole month?" which might get "around the middle" which resolves to April 12-18. Three conversational exchanges produced a date range without ever showing a date picker or asking the user to commit to specific dates up front.

This works because travel planning is inherently fuzzy. People do not start with exact dates. They start with vague ideas that crystallize through conversation. The AI should match that fuzziness instead of demanding precision from the start.

Graceful degradation

Sometimes the agent cannot find what the user wants. The search returns no results. The destination is not in the database. The request is outside the agent's capabilities. How the agent handles these moments defines the quality of the experience.

Bad degradation: "No results found." A dead end. The user has to figure out what to do next on their own.

Acceptable degradation: "I could not find any direct flights from Omaha to Dubrovnik in March. Would you like me to check flights with one stop?"

Good degradation: "There are no direct flights from Omaha to Dubrovnik. The best route I can find is through Chicago with a stop in Munich, arriving the next morning. Another option: fly out of Kansas City instead, which opens up a connection through Frankfurt with a shorter total travel time. Want me to show you those options?"

The difference is that good degradation treats the failure as a conversation, not an error. The agent explains what it tried, why it did not work, and what alternatives exist. The user never hits a wall. They hit a fork.

We design graceful degradation for the most common failure scenarios in travel booking:

No flights available for the requested route and dates. Offer nearby dates, alternative airports, or connecting itineraries.

No hotels available in the requested area and dates. Offer hotels in adjacent neighborhoods or dates with availability.

Price exceeds the user's stated budget. Acknowledge the constraint and present the closest options. "The cheapest option for that route is $820, which is above your $700 budget. Want me to check different dates where it might be cheaper?"

Feature outside the agent's capabilities. "I cannot book car rentals yet, but I can suggest rental agencies near your hotel. Want me to look that up?"

Each degradation path keeps the conversation moving forward. The user never needs to restart from scratch or go elsewhere for help.

The over-confidence problem

Over-confident AI is actively dangerous in travel. If the AI confidently tells a user they do not need a visa for Thailand when they actually do, the user could be turned away at immigration. If the AI confidently states a hotel has free cancellation when it doesn't, the user could lose money. If the AI confidently claims a specific baggage allowance that is wrong, the user could pay overage fees at the airport.

Over-confidence happens when the AI generates authoritative-sounding text based on pattern matching rather than verified data. It "sounds right" without being checked against a real source. This is the fundamental challenge of language models: they are designed to produce fluent, confident-sounding text regardless of factual accuracy.

We mitigate over-confidence in several ways.

For factual claims about bookings (prices, times, policies), the agent draws from live API data, not from generated text. When the agent says "this flight costs $487," that number came from a real-time search result, not from the language model's training data. When it says "free cancellation within 24 hours," that came from the fare rules, not from a generalization about cancellation policies.

For domain knowledge (visa requirements, travel advisories, airport information), the agent uses verified databases and flags when information might be outdated. "Based on current information, US citizens do not need a visa for stays under 30 days in Thailand. I'd recommend verifying with the Thai embassy as policies can change."

For subjective questions (best neighborhood, best time to visit, restaurant recommendations), the agent frames responses as opinions and recommendations rather than facts. "Most travelers to Tokyo recommend Shinjuku for first-time visitors because of the transit access" is better than "Shinjuku is the best neighborhood in Tokyo."

The hedging is calibrated to the stakes. For a casual restaurant recommendation, light hedging is fine. For visa requirements that could result in being denied entry, strong hedging with a recommendation to verify independently is appropriate. The level of caveat matches the cost of being wrong.

Designing uncertainty indicators

Visual and verbal cues communicate the agent's confidence level. We use both.

Verbal cues are in the text itself. "I found three flights for you" is confident. "I think these three flights might work, but let me know if you want me to look at different options" is less confident. "I'm not finding great options for that route. Here is what is available, but you might want to consider..." is openly uncertain.

The language is natural, not formulaic. We do not use confidence scores or probability labels ("87% confident"). Users do not know how to interpret those numbers. Instead, the agent uses the same hedging language that a human expert would use. A travel agent does not say "I am 73% confident this is the right hotel." They say "I think you would really like this place, though it is a bit further from the center than you usually prefer."

For structured data like flight and hotel results, confidence is communicated through completeness and context rather than labels. When the agent is confident in its results, the cards appear with clear "why this option" labels. When the results are imperfect (the agent had to relax some constraints to find options), the cards appear with honest context: "I couldn't find direct flights under $800, so here are the best one-stop options in your price range."

System-level uncertainty gets its own treatment. When an external data source is temporarily unavailable, the agent says so explicitly rather than pretending the information does not exist. "I'm having trouble checking visa requirements right now. I'll try again in a few minutes, or you can check [country]'s embassy site."

How competitors handle (or hide) uncertainty

Most travel AI implementations take the "always confident" approach. Expedia's AI trip planner generates suggestions with no uncertainty markers. Booking.com's AI responds to every query with smooth, confident prose. If the AI does not have good information, it generates plausible-sounding filler.

This works until it doesn't. Users who rely on a confident AI response and then discover it was wrong have a trust reaction that is disproportionate to the error. The anger is not just about the mistake. It is about the deception. "It told me I didn't need a visa" feels worse than "it said it wasn't sure about the visa." The first feels like a lie. The second feels like honesty.

Some general-purpose AI assistants have gotten better at this. They caveat their travel advice with "I'd recommend checking this directly" and "my training data may be outdated." But these caveats are generic, applied uniformly whether the information is likely accurate or likely wrong. They add noise without adding signal.

The best approach we have seen from competitors is Google's. When Google Flights does not have information, it simply does not display it. Missing baggage data shows as "check with airline" rather than a guess. This is honest but limited. It works for structured data where absence is detectable. It does not work for conversational AI where the model always generates something.

Uncertainty as a trust builder

Here is the counterintuitive principle: showing uncertainty builds more trust than hiding it.

When a user sees the AI acknowledge "I'm not sure about this" and then later sees the AI say "here is your flight, confirmed and booked," the contrast makes the confident statement more believable. The user has seen that the agent admits uncertainty when it exists. So when it does not express uncertainty, the information must be solid.

An AI that is always confident gives the user no way to calibrate trust. Is this one of the things it knows for certain, or one of the things it is guessing about? Without uncertainty signals, every response is equally suspect.

We see this in user behavior. Users probe AI agents with edge cases to test reliability. "What is the baggage limit for my specific fare class on this specific airline?" If the AI responds confidently with the wrong answer, the user loses trust broadly. If the AI responds with "let me check that specific fare class" and then returns the verified answer (or says "I'd recommend checking directly with the airline for this fare class"), the user's trust increases.

Honesty about limits is a feature, not a weakness. We designed it into the product because we believe it is the right approach for an AI agent that handles financial transactions. When real money is involved, the user needs to know what the AI knows for certain versus what it is guessing. Our job is to make that distinction clear, natural, and helpful rather than hiding it behind a mask of uniform confidence.

Nowah is an AI travel agent that searches and books real flights and hotels through conversation — no filters, no thirty open tabs. Plan your next trip.