Tool Use in LLMs — From Theory to Production

Function calling changed what large language models are. Before it, LLMs generated text. After it, they could interact with the world. This is not a small upgrade. It is a category shift from language tools to action systems.

We ship over a large set of tools in production at Nowah. They cover the full travel lifecycle: searching flights, ranking hotels, booking reservations, processing payments, managing itineraries, handling disruptions. I want to share what we have learned about taking tool use from a research capability to a production system that handles real bookings for real travelers every day.

Before function calling

In 2022, if you wanted an LLM to call an API, you had to get creative. The standard approach was prompt engineering: you would describe the desired output format in the model instructions, ask the model to produce a JSON blob, and then parse the result.

This worked roughly 60% of the time.

The other 40% produced invalid JSON, missing fields, hallucinated parameter values, or responses that ignored the formatting instructions entirely. You would build elaborate regex parsers to extract usable data from free text. You would retry on failure, sometimes three or four times per request. The latency was terrible. The reliability was worse.

For a travel booking system, 60% accuracy is not close to acceptable. A wrong date in a flight search wastes the user's time. A wrong passenger name on a booking creates a problem that can cost hundreds of dollars to fix. You cannot build a product that handles real money on a foundation that fails four times out of ten.

Some teams tried fine-tuning models to improve structured output. Others built intermediate parsing layers that attempted to fix malformed JSON before passing it downstream. Both approaches added complexity without fundamentally solving the reliability problem.

The core issue was that text generation and structured output are different tasks. Asking a model optimized for natural language to reliably produce machine-readable data structures is like asking a novelist to write SQL. They might get it right sometimes, but it is not what they were built for.

The function calling paradigm

When major model providers introduced native function calling in 2023, the accuracy jumped from roughly 60% to over 95% for well-defined schemas. That 35-percentage-point improvement unlocked the entire category of AI agents.

Here is how function calling works at a high level. You define a set of tools as structured schemas: name, description, parameters with types and constraints. The model receives these schemas alongside the conversation. When the model determines that a tool should be called, it produces a structured function call with the correct parameters. The system executes the call, returns the result, and the model incorporates it into its response.

The key difference from the old approach is that the model is trained to produce function calls as a native output type, not as free text that happens to look like a function call. The model understands the schema. It respects type constraints. It handles required versus optional parameters. It can even chain multiple calls when a task requires sequential steps.

For our flight search tool, the schema defines parameters like origin airport, destination, departure date, return date, number of passengers, cabin class, and preference flags. When a user says "find me flights from San Francisco to Tokyo, departing April 3rd, returning April 10th, just me in economy," the model produces a clean function call with SFO as origin, NRT or HND as destination, the correct dates, one passenger, and economy class. The success rate for this kind of well-defined extraction is consistently above 95%.

Designing a tool suite for travel

We went from zero tools to over 70. That number was not planned in advance. It grew from a core principle: every distinct action the agent might need to take should be its own tool with its own schema.

Early on, we debated whether to build a few general-purpose tools or many specific ones. A general-purpose "search" tool could handle flights, hotels, and activities through different parameter combinations. A specific approach would have separate flight search, hotel search, and activity search tools.

We chose specificity, and I think it was the right call. Here is why.

Schema clarity improves accuracy. A flight search schema with parameters like origin, destination, and departure_date is unambiguous. A generic search schema with a "type" parameter and a flexible "query" object is harder for the model to populate correctly. The more precisely the schema maps to the domain, the fewer errors the model makes.

Error handling is tool-specific. When a flight search returns no results, the appropriate recovery is to suggest nearby dates or alternative airports. When a hotel search returns no results, the recovery is to expand the neighborhood radius or adjust star rating. A general-purpose tool cannot have this specificity.

Descriptions guide selection. Each tool's description tells the model when to use it. "Search for available flights between two airports on specific dates" is clear. The model can match user intent to the right tool reliably. Vague descriptions lead to wrong tool selection.

The tradeoff is that 70+ tools create a large schema set that the model has to reason over. We manage this by organizing tools into categories and providing clear descriptions that help the model quickly identify which tools are relevant to the current request. In practice, the model handles this well. For a typical query, it considers 3-7 tools out of the full suite.

When tools fail mid-chain

Here is a scenario we deal with regularly. The user asks for flights to Tokyo in April. The agent calls the flight search tool. The search returns results. The agent picks the top three and presents them. The user selects option B. The agent calls the booking tool. The booking tool returns an error: the fare is no longer available.

What happens next defines the quality of the agent.

A naive implementation would return the error to the user: "Sorry, that fare is no longer available." The user is stuck. They have to start over.

Our agent recovers. It re-searches the same route with the same parameters. It checks whether the specific flight is still available at a different fare class. If it is, it presents the updated price and asks for confirmation. If the flight itself is gone, it finds the closest alternatives and explains what changed.

This recovery logic is not magic. It is engineering. Every tool has a defined set of failure modes, and each failure mode has a recovery strategy. API timeouts trigger retries with exponential backoff. Inventory-gone errors trigger re-search. Payment failures trigger idempotency checks to prevent double-booking. Rate limit errors trigger queuing with user notification.

The agent processes 3 to 7 tool calls per complex query. In a multi-step chain, any link can fail. The system's resilience comes from treating failure as a normal operating condition rather than an exception.

Autonomy calibration

Not all tool calls are equal. Searching for flights is low-stakes and reversible. Booking a flight and charging a credit card is high-stakes and irreversible. The agent needs different confirmation behavior for different tool categories.

We use a framework with four levels:

Auto-execute: Low-stakes, easily reversible actions that add friction if confirmed. Searching flights. Checking hotel availability. Looking up visa requirements. The agent calls these tools without asking.

Inform and proceed: Medium-stakes actions where the user should know what is happening but does not need to approve each step. Narrowing results based on preferences. Filtering out airlines the user has blacklisted. Applying loyalty program numbers.

Confirm before executing: High-stakes, irreversible actions. Booking a flight. Making a hotel reservation. Processing a payment. The agent presents the full details and waits for explicit approval.

Escalate: Actions the agent cannot handle or is not confident about. Unusual routing. Ambiguous instructions. Policy edge cases. The agent flags these for human review.

This framework is not static. As the agent demonstrates competence with a specific user, the confirmation thresholds can relax. A user who has completed 10 successful bookings might not need the same level of confirmation as a first-time user. We call this progressive autonomy, and it is one of the areas where agentic memory and tool use intersect.

Benchmarking tool-use accuracy

We track several metrics to evaluate our tool suite in production.

Tool selection accuracy measures whether the agent picks the right tool for the user's intent. If a user asks about hotel availability and the agent calls the flight search tool, that is a selection error. Our rate is above 97%.

Parameter extraction accuracy measures whether the tool receives correct values. The user said April 3rd but the agent passed April 30th. Our rate is above 95%, with most errors occurring on ambiguous inputs like relative dates ("next Friday") or informal location references ("near the city center").

Chain completion rate measures whether multi-step tool chains complete successfully end-to-end. A user asks for flights, selects one, and wants to book. The chain is: search, present, book. If any step fails and recovery also fails, the chain is broken. We track completion rate by chain length, and predictably, longer chains have lower completion rates. Three-step chains complete above 92%. Seven-step chains are closer to 85%.

Recovery success rate measures how often the agent recovers from a tool failure without the user having to intervene. We target above 80% for recoverable failure types.

The cost of inference dropped roughly 10x per year since 2023, which has a direct impact on tool use economics. Each tool call adds tokens: the schema, the parameters, the result. A 7-tool chain might consume 10-20x the tokens of a simple text response. At 2023 prices, this was expensive. At 2025 prices, it is economical.

The frontier: what comes next

Three developments are extending what tool use can do.

Parallel tool calls allow the agent to invoke multiple tools simultaneously when there are no dependencies between them. Search flights and hotels at the same time. Check visa requirements while searching for flights. This reduces latency by executing independent tasks concurrently rather than sequentially.

Tool chaining with intermediate reasoning lets the agent process the output of one tool call before deciding whether and how to call the next. This is more sophisticated than simple sequential execution. The agent reasons about partial results and adjusts its strategy mid-chain.

Self-correcting tool use is the ability of the agent to detect when a tool call produced unexpected results and automatically adjust. The flight search returned no results because the date was a Tuesday and this route only operates Monday-Wednesday-Friday. The agent recognizes the issue from the error response, shifts the date to Monday, and retries without user intervention.

First-token streaming latency is now under 500 milliseconds for frontier models. That means the user sees the agent working almost instantly, even when a complex tool chain is executing in the background. The perception of speed matters as much as actual speed, and streaming makes multi-tool interactions feel responsive.

We started with the premise that function calling turned LLMs from text generators into real-world operators. After shipping 70+ tools in production, I would refine that statement. Function calling gave LLMs the ability to act. The engineering around error handling, autonomy calibration, and chain management determines whether those actions are reliable enough for the real world.

The tool description problem

One of the less obvious challenges in production tool use is writing good tool descriptions. The model uses these descriptions to decide when to call each tool. A vague or misleading description leads to wrong tool selection.

Early in our development, we had a tool described as "Search for travel options." The model would call it for flight searches, hotel searches, activity searches, and even visa queries. The description was too general.

We rewrote it as "Search for available flights between two airports on specific dates. Returns real-time pricing and schedule data." Now the model knows exactly when to use it: when the user wants flights between specific airports on specific dates. Hotel searches go to the hotel search tool. Visa queries go to the visa tool.

Description quality has a measurable effect on tool selection accuracy. Well-written descriptions push accuracy from 93-94% to 97-98%. The effort of writing precise descriptions for 70+ tools was tedious but paid for itself many times over in reduced errors.

We also discovered that tool descriptions need maintenance. When the underlying API changes behavior, or when we add new capabilities, the descriptions need updating. A description that says "returns up to 50 results" when the API was updated to return up to 100 creates a subtle inconsistency that the model might propagate to the user.

Monitoring tool use in production

Production monitoring for tool use goes beyond standard API monitoring. We track:

Tool call volume per tool. Which tools are called most frequently? Unexpected changes in volume can indicate model behavior shifts after updates.

Parameter distribution. What values are being passed to each tool? If the origin parameter suddenly shows a lot of invalid airport codes, something changed in the extraction logic.

Error rate by tool. Which tools fail most often? High error rates might indicate a schema issue, an API problem, or a misunderstanding about when the tool should be called.

Tool call sequences. What are the common multi-tool patterns? Search-then-rank-then-present is expected. Search-then-search-then-search suggests the model is struggling to find what it needs.

User intervention rate. How often does a user have to correct the agent after a tool call? "That is not what I asked for" after a search indicates either wrong tool selection or wrong parameter extraction.

This monitoring forms a feedback loop. Production data reveals problems. Problems become evaluation scenarios. Evaluation scenarios drive prompt and schema improvements. Improvements reduce production errors.

The frontier: parallel tool calls and self-correction

The evolution of tool use is not finished. Several capabilities on the frontier will change how agents operate.

Parallel tool calls allow the model to invoke multiple tools simultaneously within a single reasoning step. When a user says "find me flights and hotels in Rome for next week," the agent can call the flight search API and the hotel search API at the same time. This halves the latency for queries that involve independent data sources.

We already support parallel tool calls in production for specific patterns: flight-plus-hotel searches, multi-leg flight queries where legs are independent, and simultaneous price checks across fare classes. The latency improvement is significant. A sequential flight-then-hotel search that took 4 seconds now completes in 2.5 seconds.

Self-correcting tool use is when the model recognizes that a tool call failed or returned unexpected results and automatically adjusts. The flight search returns zero results because the dates fall on a holiday with sold-out inventory. Instead of reporting "no flights found," the agent recognizes the likely cause and widens the date range: "No flights available on December 25th. Here are options for December 24th and 26th."

This self-correction loop is subtle. The model needs to distinguish between "no results because the parameters are wrong" (fix the parameters) and "no results because the query is impossible" (tell the user). We implement heuristics for common failure patterns: zero results on peak travel dates suggests date flexibility, while zero results on an obscure route suggests the route does not exist.

Tool chaining with planning is the combination of tool use and multi-step planning. The agent decides it needs to call tool A, use the results to determine parameters for tool B, and combine both results for tool C. This is how complex bookings work: search flights, use the arrival time to determine hotel check-in, use the hotel location to find nearby restaurants.

Current models handle 3-4 step chains reliably. Longer chains introduce more opportunities for error accumulation. We mitigate this through checkpoint verification: after every 2-3 tool calls, the agent summarizes what it has found and confirms the direction before continuing.

Cost optimization for tool-heavy interactions

Each tool call adds tokens to the conversation: the tool schema, the call parameters, and the result payload. A flight search might return 2,000-3,000 tokens of result data. Multiply by 5-7 tool calls in a complex query, and the token cost per interaction is substantial.

We optimize cost through several strategies.

Result compression. Tool results are summarized before being added to the context. A flight search that returns 50 results does not add all 50 to the conversation. The ranking layer selects the top 3-5 and adds only those.

Schema caching. Tool schemas do not need to be re-sent with every model call in a conversation. We cache them in the model instructions and reference them by name.

Selective tool inclusion. Not all 70+ tools need to be in the schema for every conversation. A conversation that is clearly about flights does not need the hotel-related tools in its schema. We dynamically select a relevant subset based on the conversation context.

Result caching. If the user asks a follow-up question about results the agent already retrieved, we serve from cache rather than making a new API call.

The cost of inference dropped roughly 10x per year since 2023, making these optimizations less critical than they once were. But at scale, even small per-interaction savings compound significantly.

The evolution is not over

Tool use in LLMs has improved dramatically in the past three years, but it is still evolving. The next capabilities on the horizon will further close the gap between what agents can do and what users need.

Better error messages from failed tool calls will help models self-correct more reliably. Standardized tool description formats will reduce the bespoke schema engineering that each product team currently does independently. Improved model understanding of tool result semantics will reduce the cases where the model misinterprets returned data.

We also expect tool use to become more cost-efficient. Today, including 70+ tool schemas in the model instructions consumes significant tokens. Future models may support external tool registries that do not require the full schema in every prompt, reducing per-interaction costs substantially.

For AI travel booking, reliable tool use is not optional. It is the entire product. An agent that can talk about flights but not book them is a chatbot. An agent with a comprehensive, well-tested tool suite that searches, ranks, books, and manages is the best travel app we know how to build.

Nowah is an AI travel agent that searches and books real flights and hotels through conversation — no filters, no thirty open tabs. Plan your next trip.