Rate Limiting an AI Travel Agent Without Ruining the Experience

One misbehaving client burned through $500 in language model inference in 10 minutes. It was a bug, not an attack. A retry loop in a client's network layer kept re-sending the same message. Each message triggered a full AI processing cycle: reasoning, tool calls, flight searches, response generation. At roughly $0.05-0.50 per query depending on complexity, those retries added up fast.

Rate limiting for AI applications isn't primarily about preventing DDoS attacks (though it helps with that too). It's about cost control. Every message to our AI agent triggers inference that costs real money. Unlike serving a static web page, where the marginal cost of an additional request is negligible, an additional AI query has meaningful compute cost. Rate limiting is the mechanism that keeps those costs bounded.

Cost-aware endpoint categorization

Not all endpoints cost the same to serve. A request to fetch a user's trip list hits the database and returns JSON. Fast, cheap, simple. A request to the AI chat endpoint triggers language model inference, potentially multiple tool calls to external APIs, and a streaming response. Slow, expensive, complex.

We set different rate limits for different endpoint categories based on their cost and risk profiles:

General API gets 100 requests per 15 minutes. Most CRUD operations fall here. The limit is generous because the per-request cost is low.

Authentication gets 10 attempts per 15 minutes. Tight, because auth endpoint abuse is a brute force attack vector and each failed attempt generates logging and processing overhead.

AI chat gets 30 messages per minute. This is the most expensive endpoint. Each message triggers inference that costs 10-100x more than a database query, depending on complexity. The limit balances usability (a traveler shouldn't hit the limit during normal conversation) with cost control (a misbehaving client can't generate unbounded inference cost).

Booking gets 5 per hour. Intentionally low because each booking attempt interacts with payment processors and travel data providers, both of which have their own rate limits and costs.

Payments get 10 per minute. Payment operations are sensitive and relatively expensive due to payment processor API calls.

Sliding window rate limiting

We use a sliding window algorithm rather than fixed windows. The difference matters for user experience.

A fixed window resets at a specific time. If the window resets at the top of each minute and you send 30 messages at 12:00:59, you could send another 30 at 12:01:01. Two seconds apart, 60 messages. That's not really 30 per minute.

A sliding window counts requests in a rolling time period. It doesn't matter when within the period you send them. The last 60 seconds always contain at most 30 messages. This is smoother and more predictable.

The implementation uses our cache layer. Each request increments a counter with a key that includes the user ID and a time component. The counter has a TTL matching the window duration. The sliding window algorithm interpolates between the previous and current window counts to approximate a true rolling window.

Tiered limits

Not all users should have the same limits. A traveler who's actively planning a complex multi-city trip generates more AI queries than someone checking on a simple one-way flight. A power user who travels weekly needs more headroom than a casual vacation planner.

We support tiered rate limits based on subscription level. Free users get the base limits. Premium subscribers get higher thresholds. The specific multipliers are configured in the rate limiting middleware and can be adjusted without a deployment.

The tier check is cheap. It's a lookup from the user's profile, which is already loaded as part of authentication. No additional database query. The rate limit middleware checks the user's tier and applies the appropriate threshold.

Graceful error messages

Here's something I feel strongly about. A rate-limited request should never return a cryptic error. "Error 429" tells the user nothing useful. "Too many requests" is slightly better but still unhelpful.

Our rate limit responses include:

What happened. "You've sent too many messages in a short time."

When they can retry. A `Retry-After` header with the number of seconds until the limit resets, plus a human-readable message: "You can send another message in 45 seconds."

Why it happened. For AI endpoints: "Each message involves complex processing. The limit helps us maintain quality for all travelers."

The goal is for the traveler to understand the situation, know exactly when they can continue, and not feel punished. Rate limiting protects the system, but the user experience of hitting a limit should be informative, not hostile.

Monitoring and threshold tuning

Rate limits set too low frustrate legitimate users. Limits set too high don't prevent abuse. Finding the right thresholds requires monitoring.

We track:

Limit hit rate per endpoint category. What percentage of users hit the rate limit during normal usage? If more than 1-2% of legitimate users regularly hit the limit, the limit is probably too low.

Abuse detection rate. Of the requests that get rate-limited, how many come from patterns that look like actual abuse (automated scripts, retry loops, scraping) vs. legitimate heavy usage?

Cost per user. What's the average AI inference cost per user per day? If this exceeds our unit economics target, rate limits might need to be tighter, or we need to look at optimizing inference costs.

We adjust thresholds quarterly based on this data. The adjustments are conservative. We'd rather slightly over-limit than under-limit because the cost of an AI inference spike is immediate and significant.

Rate limits are defense in depth

Rate limiting is not our primary security mechanism. Authentication and authorization come first. A rate-limited unauthenticated request is still unauthenticated. Rate limiting prevents the volume-based attacks that slip past auth: credential stuffing, API scraping, and the runaway client bugs that are more common than intentional attacks.

Think of it as the last layer before cost becomes unbounded. Authentication verifies identity. Authorization verifies permissions. Rate limiting verifies that the volume of requests is within expected bounds. All three are needed.

If you're building an AI product with meaningful per-request costs, set rate limits from day one. Don't wait for the runaway client bug to teach you why they matter. The $500 lesson we learned was cheap. Others have learned it at much higher cost.

Nowah is an AI travel agent that searches and books real flights and hotels through conversation — no filters, no thirty open tabs. Plan your next trip.