--- title: Rate Limiting for AI-Driven Traffic Patterns description: "AI agents generate bursty, multi-call traffic unlike human clicks. Here is how we protect upstream APIs and manage costs with AI-aware rate limiting." canonical: https://nowah.xyz/blog/rate-limiting-ai-driven-traffic lastModified: "2026-08-07T03:49:16.451Z" --- # Rate Limiting for AI-Driven Traffic Patterns AI agents generate bursty, multi-call traffic unlike human clicks. Here is how we protect upstream APIs and manage costs with AI-aware rate limiting. [Rate limiting](/blog/rate-limiting-ai-agent-experience) is one of those engineering topics that sounds boring until you realize that bad rate limiting either breaks your product or drains your bank account. For an AI travel platform, the stakes are higher on both sides. Our AI agent generates traffic patterns that look nothing like human click-based traffic, and every unnecessary API call to a travel data provider costs real money. ## AI traffic is bursty and multi-call ![Illustration for this section](https://pics.nowah.xyz/website-media/engineering-040-img-1.webp) When a human uses a traditional travel website, the traffic pattern is predictable. They load a page. They look at it for 10-30 seconds. They click something. Another page loads. The requests are evenly spaced and each one is independent. When our AI agent handles a user request, the pattern is completely different. The user sends one message. The agent immediately makes 3-5 internal calls: check user preferences, search for flights (which might fan out to multiple providers), rank results, format the response. All of this happens in a 5-10 second burst. Then silence until the next user message. This bursty pattern means our peak traffic within a single second is much higher than our average traffic per second. Traditional rate limiters that use fixed windows ("100 requests per minute") [struggle with](/blog/legacy-otas-struggle-with-ai-architecture) this because a burst of 30 requests in one second is fine for us but would be flagged by a per-second limit of 10. We use sliding window rate limiting instead of fixed windows. A sliding window smooths out bursts by looking at a trailing time period rather than fixed minute boundaries. This allows natural bursts while still preventing sustained abuse. ## Per-route rate limits Not all API calls are equal. A chat message costs CPU time but no external API cost. A flight search costs $0.03-0.10 per provider call. A booking attempt costs nothing if it fails but creates a financial transaction if it succeeds. We set different rate limits for different routes based on their cost and risk profile: General API access: 100 requests per 15 minutes. This covers most read operations. Chat messages: 30 messages per minute. This prevents runaway conversations that could generate excessive AI inference costs. Booking attempts: 5 per hour. This prevents accidental or malicious rapid booking attempts. Payment operations: 10 per minute. High enough for legitimate use, low enough to catch anomalies. These limits are per-user, identified by authenticated session. Unauthenticated requests get much tighter limits. ## Protecting upstream travel APIs ![Supporting diagram](https://pics.nowah.xyz/website-media/engineering-040-img-2.webp) Our rate limits are not just about protecting our own systems. They also protect our relationship with travel data providers. Each provider has their own rate limits, and exceeding them risks throttling or temporary bans. If our provider allows 50 searches per second and we have 60 users simultaneously searching, we need to queue requests, not blast them all through. We implement a provider-level rate limiter that sits between our AI agent and the [external APIs](/blog/circuit-breakers-external-apis). When the agent requests a flight search, it enters a provider-specific queue. If the provider is at capacity, the request waits. The agent receives a "search in progress" signal and streams an appropriate status message to the user ("Checking availability, one moment..."). This queuing is invisible to users most of the time. Under normal load, searches go through immediately. Under high load, there is a brief additional wait. The key is that the wait is handled gracefully with real-time status updates rather than a timeout error. ## Graceful handling in conversation When a user hits a rate limit, traditional apps show an error page or a "too many requests" message. That does not work in a conversation. Our AI agent handles rate limits conversationally. If the chat rate limit is hit, the agent says something like "I'm handling a lot right now. Give me a moment and I'll get back to you." If a search rate limit is hit, the agent queues the search and lets the user know: "Flight searches are taking a bit longer than usual. I'll have your results shortly." This conversational handling turns a frustrating technical limitation into a reasonable human interaction. The user does not see an error code. They see an agent that is busy but responsive. ## Cost-based rate limiting Beyond protecting systems and providers, we also use rate limiting as cost management. AI inference and travel API searches have real per-request costs. A single user having an unusually long conversation with many search refinements can generate significant cost. We track cost-per-session in real time. If a single session exceeds a cost threshold, we reduce the frequency of expensive operations (new searches) while keeping cheap operations (text responses, questions, conversation) unrestricted. The agent pivots to asking clarifying questions and referencing already-fetched results instead of running new searches for every minor refinement. Caching and request reduction cut our costs by a large share. Smart caching means that when a user says "what about a day later?" the agent adjusts the search parameters and checks if cached results cover the new dates rather than running an entirely new provider search. When ten users search for similar routes on the same day, the second through tenth searches hit cache instead of the provider API. Rate limiting is infrastructure that users should never notice when it is working well. The goal is invisible protection: the system stays fast, the costs stay bounded, and the travel data providers stay happy, all without any user ever seeing a "too many requests" error. --- Nowah is an AI travel agent that searches and books real flights and hotels through conversation — no filters, no thirty open tabs. [Plan your next trip](https://app.nowah.xyz).