--- title: "When AI Gets It Wrong: Our Approach to Error Recovery" description: "AI will make mistakes. The question is not if but how often and how gracefully. Detection, recovery, honesty, and prevention form our error strategy." canonical: https://nowah.xyz/blog/error-handling-recovery lastModified: "2026-08-07T07:55:35.731Z" --- # When AI Gets It Wrong: Our Approach to Error Recovery AI will make mistakes. The question is not if but how often and how gracefully. Detection, recovery, honesty, and prevention form our error strategy. Last month, a user asked our AI agent to find flights from San Francisco to Barcelona. The agent searched, found good options, and presented three flights. One problem: the user had previously mentioned they were based in San Jose, not San Francisco. The agent picked the closest major airport instead of the user's actual airport. The $40 Uber to SFO matters. This was not a catastrophic failure. The flights were real, the prices were accurate, the dates were correct. But it was wrong, and the user noticed. What happened next is what I want to talk about, because how an AI product handles its mistakes matters more than how often it makes them. ## AI will make mistakes ![Illustration for this section](https://pics.nowah.xyz/website-media/product-070-img-1.webp) I am going to start with something that AI marketing departments hate to say: AI agents will get things wrong. Regularly. Not because the technology is bad, but because natural language is ambiguous, user intent is sometimes unclear, and real-world data is messy. Our agent misunderstands requests about 7% of the time. That means roughly 1 in 14 interactions starts with the agent heading in a slightly wrong direction. The severity varies from minor (wrong airport, fixable in seconds) to significant (wrong destination entirely, requiring a conversation reset). Anyone who tells you their AI product does not make mistakes is either lying or not measuring. The honest starting point, and the one that actually builds user trust, is acknowledging that errors are part of the system and building to handle them well. This is counterintuitive for product teams. The instinct is to minimize the perception of errors. Hide the mistake. Pretend it did not happen. Redirect the user without acknowledging what went wrong. We take the opposite approach, and the data supports us. ## Categories of errors Not all AI errors are created equal. We categorize them because different types require different handling: **Misunderstood intent.** The agent understood the words but not what the user meant. "Something close to the beach" might mean "within walking distance" to the user but "within a 20-minute drive" to the agent. These are the most common errors and usually the easiest to fix through a follow-up question. **Wrong results.** The agent understood the intent but returned incorrect information. A hotel that is listed as having a pool but does not. A flight price that has changed between search and booking. These are data errors, and they erode trust faster than intent errors because the user trusted the agent's output and it was wrong. **Booking failures.** The user selected an option and started the booking flow, but something went wrong: payment declined, flight no longer available, hotel sold out between search and booking. These are the highest-stress errors because the user has already committed mentally and emotionally to the booking. **Hallucinated information.** The agent states something confidently that is simply not true. "This airline includes two free checked bags" when it does not. "The visa on arrival for Thailand is valid for 60 days" when the policy recently changed. These are the most dangerous errors because they can lead to real-world consequences: showing up at the airport with too many bags, or arriving in a country without the right visa. Each category demands a different response strategy. Misunderstood intent needs clarification. Wrong results need correction. Booking failures need recovery paths. Hallucinations need detection before they reach the user. ## Detection methods The best error is one caught before the user notices. We invest heavily in detection because prevention is always cheaper than recovery. **Confirmation gates.** For high-stakes actions, the agent confirms before proceeding. Before searching, it confirms the parameters: "Just to confirm: you want flights from San Jose to Barcelona, departing March 20, returning March 28?" This catches misunderstood intent before a search even runs. We calibrate confirmation frequency based on the agent's confidence. High-confidence interpretations proceed without confirmation. Low-confidence ones get a check. **Consistency checks.** When the agent returns results, we run automated checks against the conversation context. If the user said "budget-friendly" and all three hotel options are over $300/night, something is off. If the user requested direct flights and the results include connections, the tool parameters were wrong. These checks flag potential errors before results reach the user. **User signals.** After the agent responds, user behavior provides real-time error detection. A long pause followed by a rephrase is a strong signal that the response was wrong or unhelpful. Selecting "none of these" or asking to see different options suggests curation failure. These signals do not prevent the error but they detect it quickly enough for in-conversation recovery. **Factual verification.** For informational responses ([visa requirements](/blog/ai-agents-visa-requirements-documents), baggage policies, airport information), we cross-reference the agent's output against our knowledge base. If the agent states something that contradicts our structured data, we flag it and either correct the response before it ships or add a caveat. No detection system catches everything. Our goal is to catch the most consequential errors, especially booking-related mistakes and factual hallucinations, before they affect the user. Less consequential errors (suboptimal curation, slightly off tone) we catch through implicit signals and fix in subsequent interactions. ## Graceful recovery Back to the San Francisco vs. San Jose example. Here is what the recovery looked like: The user said: "I'm in San Jose, not SF." The agent responded: "You're right, sorry about that. Let me search again from San Jose International. The flight options will be slightly different since SJC has fewer direct international routes." Then it re-searched, found new options, and presented them with a note: "These depart from SJC. There's one direct option and two with connections. The direct flight is $80 more than what I showed from SFO, but it saves you the trip to San Francisco." Three things happened here that matter: **Acknowledgment.** The agent did not pretend the error did not happen. It did not silently re-search and hope the user would not notice. It said "sorry about that." Simple, direct, no over-apologizing. **Explanation.** The agent explained why the error had consequences ("fewer direct international routes from SJC"). This is not just politeness. It provides useful context that helps the user evaluate the new options. **Recovery within the conversation.** The user did not get sent to an error page. They did not have to restart the conversation. The correction happened right in the flow, and the conversation continued naturally. We have A/B tested recovery styles. The transparent recovery approach (acknowledge, explain, fix) consistently outperforms the silent recovery approach (just re-search without mentioning the error). Users who experience transparent recovery report higher satisfaction than users who experience silent recovery, and in some cases higher satisfaction than users who never experienced an error at all. This makes psychological sense. When someone acknowledges a mistake and handles it well, it builds trust. You think "okay, if something goes wrong, they will be honest about it and fix it." When the mistake is silently glossed over, you think "what else are they getting wrong that I am not catching?" ## The honesty principle We have a rule for the agent: when uncertain, say so. Do not bluff. This sounds obvious, but it runs counter to how most AI systems are designed. Language models are trained to produce confident-sounding responses. They do not naturally say "I'm not sure" or "I should double-check that." Left to its default behavior, the model will state uncertain information with the same confidence as certain information. We explicitly instruct the agent to express uncertainty when it exists. Examples: - "I believe this airline charges for carry-on bags, but policies change frequently. Let me verify that before you book." - "The weather in Bali in September is usually dry, but I would check closer to your departure date since it can vary." - "This hotel shows availability, but the last room might go quickly. Want me to proceed with the booking now?" Each of these could be stated as a confident fact. The agent could say "this airline charges for carry-on bags" without the hedge. But the hedge is honest, and honesty compounds into trust. The honesty principle also applies to the agent's limitations. If a user asks something the agent cannot do, we want a clear, direct response: "I can not book rental cars yet, but that is something we are working on. For now, I would suggest checking the rental car desk at the airport or using one of the major rental sites." This is better than a vague "I'm unable to help with that" because it acknowledges the limitation and provides an alternative. Users are remarkably forgiving of honest limitations. They are much less forgiving of confident incorrectness. ## Prevention: errors feed back into improvement Every error is data. And data, properly used, prevents future errors. When our detection systems catch an error, or when a user corrects the agent, we capture the full context: what the user said, how the agent interpreted it, what went wrong, and what the correct response should have been. This feeds into three improvement channels: **Prompt refinement.** If the same type of error occurs repeatedly, we adjust the agent's instructions to handle that pattern better. The SFO/SJC example led us to add specific guidance about defaulting to the user's stated home airport rather than the nearest major hub. **\[Eval suite\]\(/blog/evaluating\-ai\-travel\-agent\-beyond\-unit\-tests\) expansion\.** Every novel error type becomes a test case. The next time we change the agent's behavior, it is tested against this scenario. The eval suite is a growing catalog of everything that has ever gone wrong, which means the agent cannot regress on fixed issues. **Tool improvements.** Some errors stem from tool behavior rather than agent reasoning. If the hotel search tool returns stale availability data, fixing the agent's prompt will not help. The tool itself needs to refresh data more aggressively or add availability confidence indicators. The prevention loop means that a specific error type should never become a persistent problem. First occurrence: detected and recovered. Second occurrence: root cause analyzed. Third occurrence: fix deployed. This does not mean errors stop happening. New error types emerge as users interact in new ways and as the product handles new scenarios. But old error types fade. ## What competitors get wrong about errors Most AI travel products treat errors as something to minimize in their marketing. "Our AI is 99% accurate!" (Measured how? On which tasks? Accuracy of what?) The [problem with](/blog/problem-with-travel-loyalty-programs) this framing is that it sets expectations that cannot be met. A user who expects 99% accuracy is surprised and disappointed by the 1% failure. A user who expects an AI agent that is usually right, sometimes wrong, and always honest about it is pleasantly surprised by good performance and unsurprised by occasional errors. The other common mistake is building elaborate error handling infrastructure that routes users to fallback experiences. "The AI could not help, here is a traditional search form." This is the worst of both worlds. You have trained the user to interact conversationally, and now you are punting them to a form-based interface when things get hard. It signals that the AI is not trustworthy for anything beyond simple queries. Our approach is to keep the user in the conversation. The agent made a mistake? It recovers in the conversation. The search returned no results? The agent suggests alternatives in the conversation. The booking failed? The agent explains why and offers recovery options in the conversation. The user never leaves the interface they are comfortable with. Error handling is not a separate system bolted onto the product. It is integrated into the product's core interaction model. The same agent that handles the happy path handles the error path. And often, it handles the error path better than a traditional product could, because it can combine acknowledgment, explanation, and recovery in a natural, conversational way that no error page or modal dialog can match. AI will make mistakes. That is the starting point. What you do about it, detection, honesty, in-flow recovery, and systematic prevention, is what separates a trustworthy AI product from one that loses users at the first failure. We chose to build around the reality of errors rather than pretend they do not exist. So far, the users agree that is the right call. --- Nowah is an AI travel agent that searches and books real flights and hotels through conversation — no filters, no thirty open tabs. [Plan your next trip](https://app.nowah.xyz).