Model Selection for Production AI Agents

I have a confession that will surprise nobody who has shipped an AI product: we have changed our primary model four times in eighteen months. Not because we were chasing hype. Because the capability-cost-latency triangle kept shifting under our feet, and each shift opened up product possibilities that did not exist three months earlier.

Model selection for a production AI travel agent is not an academic exercise. It is an operational decision that directly affects whether your user gets a helpful response in 1.5 seconds or a mediocre one in 4. At consumer scale, that gap is the difference between a product people love and one they abandon.

The capability-cost-latency triangle

Every model sits somewhere in a three-dimensional space defined by capability (how well it reasons), cost (how much per million tokens), and latency (how fast the first token arrives). You can optimize aggressively for two of these. Optimizing for all three simultaneously remains, as of early 2026, impossible.

A frontier model gives you the best reasoning for complex multi-city itinerary planning, but it costs more per query and often has higher first-token latency. A smaller model responds in under 200 milliseconds and costs a fraction of a cent per query, but it struggles with nuanced preference matching or multi-constraint optimization.

The key insight we arrived at after months of production data: you do not need to pick one model. You need a routing strategy.

When frontier models earn their cost

Some queries genuinely need frontier-class reasoning. A user says: "I need to fly from San Francisco to Tokyo for two weeks in April, but I want to stop in Honolulu for three days on the way there, my wife is joining from Chicago and meeting me in Tokyo, we both need aisle seats, and our budget is $4,000 total."

This is a multi-constraint optimization problem. The agent needs to coordinate two separate origin cities, a stopover, seat preferences, budget constraints, and date flexibility across multiple flight segments. A smaller model will miss constraints, produce incoherent itineraries, or fail to properly orchestrate the tool calls needed to search and compare options.

For queries like this, we route to the most capable model available. The cost per query is higher, but the transaction value is also higher. When the average booking is $500 or more, spending an extra few cents on inference to get the recommendation right is trivially justified.

The other place frontier models consistently outperform is personalization. When the agent needs to weigh a user's historical preferences (always books direct flights, prefers boutique hotels near public transit, avoids airports with long immigration lines) against the available options, the quality gap between frontier and mid-tier models is measurable. We see a 15-20% improvement in recommendation acceptance rate when using frontier models for preference-heavy queries.

When smaller models win

The majority of queries in a travel AI agent are not complex. "What time does my flight leave?" "Can you show me that hotel again?" "Change it to a window seat." These are retrieval or simple modification tasks that a smaller, faster model handles perfectly.

For these, latency is the priority. A user asking about their flight departure time expects an answer in under a second, not a thoughtful three-second pause while a frontier model contemplates the nature of time zones. We route simple queries to fast models that respond with first-token latency under 200 milliseconds.

The cost savings compound quickly. If 60% of your queries can be served by a model that costs 10-20x less per million tokens, your overall inference budget drops dramatically. At consumer scale with thousands of daily conversations, this is the difference between a sustainable business and one that burns cash on every interaction.

Latency budgets for conversational AI travel booking

Here is something I rarely see discussed in model selection literature: conversational AI has fundamentally different latency requirements than batch processing or even single-query applications.

In a conversation, perceived latency matters more than absolute latency. A streaming response that starts in 400 milliseconds and completes over 3 seconds feels faster than a non-streaming response that arrives complete in 2 seconds. This is because the user sees activity immediately and interprets it as the agent working, which is exactly what is happening.

We set our latency budgets around this:

• First token: under 500 milliseconds for any query type
• Simple responses: complete in under 1.5 seconds
• Complex searches: first text acknowledgment in under 500 milliseconds, first result card within 3 seconds, full ranked results within 6 seconds
• Booking execution: confirmation within 10 seconds (this includes external payment and booking API calls)

These budgets constrain model selection directly. If a frontier model cannot reliably deliver first-token in under 500 milliseconds with streaming, it cannot be our primary model regardless of capability. AI trip planning has to feel like a conversation, not like submitting a query to a mainframe.

Model fallback strategies

Production systems fail. Models have outages. Rate limits get hit. Latency spikes happen during peak hours. If your entire product depends on a single model endpoint, you have built a fragile system.

We run a fallback chain. If the primary model is unavailable or latency exceeds our budget, the request routes to the secondary model automatically. The user never sees an error message. They might get a slightly less nuanced response, but they get a response.

The tricky part is making this seamless. Different models have different model instructions behaviors, different tool-calling conventions, and different output styles. We invested significant engineering effort in an abstraction layer that normalizes these differences so a fallback feels invisible to the user. The agent personality stays consistent even when the underlying model changes mid-conversation.

The total cost picture

Model inference cost is the number everyone fixates on, but it is only one component. A single conversational turn in our travel agent involves:

• Input tokens: the model instructions, conversation history, memory context, and user message
• Output tokens: the agent's response, including any reasoning
• Tool calls: each tool invocation is an additional inference cycle
• Memory retrieval: querying the preference and history store
• External API calls: flight search, hotel search, and other travel data APIs

For a complex query that triggers three tool calls, the total cost is roughly 3-5x the raw inference cost of a single model call. This is important for unit economics. When we model cost per interaction, we include everything, not just the LLM invoice.

The good news: inference costs have been dropping roughly 10x per year since 2023. A query that cost $0.10 in early 2024 costs about $0.001 in early 2026. This trend shows no sign of slowing. Models are getting cheaper, faster, and more capable simultaneously, which means the capability-cost-latency triangle is shrinking. The tradeoffs are getting less painful every quarter.

For a travel AI agent specifically, the economics work because the transaction values are high. When the best travel app helps someone book a $1,200 flight, even a relatively expensive AI interaction of $0.05-0.10 is a rounding error on the revenue. Compare that to a consumer chat application where the revenue per interaction might be measured in fractions of a cent. Travel's unit economics give us room to use the best models for the queries that need them.

What I would tell someone choosing today

If you are building an AI travel agent today, or any agentic system that handles financial transactions, here is what I would do:

Start with a frontier model for everything. Get the product right. Understand which queries actually need that capability and which do not. Then instrument your system to categorize queries by complexity and start routing the simple ones to cheaper, faster models.

Do not over-optimize on cost before you have product-market fit. The cost difference between frontier and budget models, at pre-scale volumes, is almost certainly less than your engineering salaries. Get the experience right first.

Build the fallback infrastructure from day one. It is much harder to retrofit than to design in. Your abstraction layer for model-agnostic tool calling and response formatting will pay dividends every time a provider has an outage or you want to evaluate a new model.

And keep your model selection modular. The landscape changes every few months. The model that is optimal today will not be optimal in six months. If swapping models requires a sprint of engineering work, your architecture has a problem.

The best flight booking experience is one where the model selection is invisible to the user. They ask a question, they get a great answer, fast. Everything else is implementation detail.

Nowah is an AI travel agent that searches and books real flights and hotels through conversation — no filters, no thirty open tabs. Plan your next trip.