5 Things to Know Before Building an AI Travel Product

Building with AI is nothing like building traditional software. I came from a world where you write a function, give it an input, and get a predictable output. Every time. That mental model will break you if you carry it into AI product development.

Here are five things I wish someone had told me before we started.

1. Non-deterministic outputs break every testing assumption you have

In traditional software, a test passes or fails. Input A produces output B. Always. If it does not, something is broken.

AI does not work this way. The same input can produce different outputs every time. Ask the agent to find flights to Tokyo and you might get three different sets of recommendations on three different runs, all of which are valid. Your test framework needs to evaluate whether the output is acceptable, not whether it is identical.

We had to rebuild our entire testing approach. Our evaluation framework uses 60-second timeouts and automatic retries because some valid responses take longer than others. Tests run sequentially to avoid rate limiting from our data providers. We evaluate accuracy within ranges, not against exact expected outputs.

The meta-lesson: if you are building an AI product and your tests look like traditional unit tests, you are doing it wrong.

2. Users anthropomorphize AI agents within seconds

The first time I watched a user interact with our agent, they said "thank you" after receiving flight options. Then they said "sorry to bother you" when asking a follow-up question. Within the first conversation, they were treating the AI like a person.

This has profound implications for product design. Tone matters. Patience matters. The agent's willingness to handle repeated questions gracefully matters. A curt or robotic response does not feel like a software limitation. It feels like rudeness.

We spend significant time on the agent's conversational behavior, not because we want to pretend it is human, but because users unconsciously expect human-like interaction patterns. If the agent interrupts, rushes, or dismisses, users feel it emotionally, not just functionally.

Design for the fact that your users will talk to your AI like a person, whether you intend that or not.

3. Latency tolerance is lower than you think

I assumed users would be patient with AI. It is doing complex reasoning, after all. Surely people understand that comparing a hundred flights takes a moment.

Wrong. Two seconds feels fine. Five seconds feels slow. Ten seconds feels broken. And this is true even when users intellectually understand that the AI is doing real work.

The solution is streaming. When users see the agent's progress in real time, "Searching 147 flights... Comparing prices across 12 airlines... Ranking by your preferences..." they are dramatically more patient. Streaming thirty seconds feels faster than a loading spinner for ten seconds because visible progress creates a perception of speed.

We target getting some visible output to the user within one to two seconds of their message. The full response can take longer, but the first indication that the agent is working must be nearly instant.

4. AI capabilities change monthly — your roadmap must flex

We built features based on what the AI model could reliably do at the time. Two months later, the model was significantly better, and some of those features could have been built more simply or ambitiously.

AI model capabilities shift on a monthly cadence. A workaround you built in January might be unnecessary by March. A feature you thought was impossible might become trivial by summer. If your roadmap is rigid, it becomes obsolete before you ship it.

We now plan in shorter cycles and maintain explicit flexibility for "what becomes possible." Every quarter, we reassess which planned features can be simplified because the underlying model got better, and which ambitious features should be moved forward because the technology caught up.

The founders building great AI products are the ones who treat model improvements as a tailwind to be surfed, not a fixed foundation to build on.

5. The hardest part is product design, not the AI itself

I expected the AI engineering to be the bottleneck. Prompt engineering, tool orchestration, memory management, evaluation frameworks. These are genuinely hard.

But the hardest decisions are all product design. How much should the agent decide versus present options? When should it ask for confirmation versus just act? How do you handle the inevitable cases where it misunderstands something? What is the right level of detail in a recommendation? How do you balance speed with thoroughness?

Product design decisions outnumber AI engineering decisions roughly three to one. And they are harder because there is no "right answer." The best response depends on context, user preference, and trust level. We are constantly tuning these balances based on user feedback.

The meta-lesson

If there is one thing that ties all five together, it is humility. Building with AI requires a constant willingness to be wrong, to be surprised, and to redesign what you thought was settled. The technology changes. User behavior defies expectations. Testing requires new paradigms. And product design is never done.

The founders who struggle with AI products are the ones who treat AI like a deterministic technology that should behave predictably. The ones who succeed are the ones who embrace the uncertainty and build systems that are resilient to it.

If you are about to start building an AI product, start with the testing framework. Get comfortable with non-determinism. Watch real people in private testing within the first week. And keep your roadmap loose enough to absorb the surprises, because they are coming, and they are the most interesting part.

Nowah is an AI travel agent that searches and books real flights and hotels through conversation — no filters, no thirty open tabs. Plan your next trip.