Circuit Breakers for External Travel APIs

The flight search API responded in 15 seconds instead of 2. Every request to our platform that triggered a flight search sat there, holding a connection, waiting. Within three minutes, the connection pool was exhausted. Within five minutes, requests that had nothing to do with flight search started failing because there were no connections available. The hotel search was fine. The chat was fine. But it didn't matter, because every request was stuck behind the flight search requests that wouldn't let go of their connections.

One slow external API took down everything. Not because the API was down (it was responding, just slowly), but because we treated a slow response the same as a normal response. We waited politely. The polite waiting nearly killed us.

The cascade failure pattern

External API slowdowns are more dangerous than external API outages. When an API is fully down, requests fail fast. The caller gets an error in milliseconds and can handle it. When an API is slow, requests pile up. Each slow request holds resources (a connection, a thread, memory) while it waits. New requests arrive and also start waiting. The resource pool fills up. Eventually, the waiting requests consume all available resources and nothing else can make progress.

Our travel data provider targets 1-3 second response times for flight search. That's the normal case. But external APIs have bad moments. Infrastructure issues, increased load from other customers, deployment-related latency spikes. When those moments happen, response times can spike to 10, 15, 30 seconds.

Without protection, each 15-second request ties up a connection for 15 seconds. If we normally handle 50 flight searches per second and each takes 2 seconds, we need about 100 connections. If response time jumps to 15 seconds, we need 750 connections for the same throughput. We don't have 750 connections. Nobody does. The pool fills up and the entire system stops.

The circuit breaker pattern

A circuit breaker sits between your code and the external API. It monitors the success and failure rate of requests and has three states.

Closed is the normal state. Requests flow through to the external API. The circuit breaker counts failures and tracks response times. Everything is working.

Open is the protection state. Too many failures have occurred. The circuit breaker stops sending requests to the external API entirely. Instead, it returns an error immediately. No waiting. No connection held. The caller knows instantly that the external service is unavailable and can handle it.

Half-open is the recovery state. After a cooling period, the circuit breaker lets a small number of requests through to the external API. If they succeed, the circuit closes and normal traffic resumes. If they fail, the circuit opens again and the cooling period restarts.

The key insight is that failing fast is better than failing slow. A fast failure takes milliseconds and frees resources immediately. A slow failure takes seconds or minutes and holds resources the entire time. The circuit breaker converts slow failures into fast failures.

Failure detection and thresholds

The threshold configuration matters. Too sensitive and the circuit opens on normal transient errors. Too lenient and the circuit doesn't open fast enough to prevent cascade failures.

We use a sliding window of the last 60 seconds of requests. If more than 50% of requests in that window fail or exceed the response time threshold, the circuit opens. The 50% threshold means we tolerate occasional failures (which are normal with external APIs) but react quickly to sustained degradation.

The response time threshold is set at 3x the normal response time. If the travel data provider normally responds in 2 seconds, the threshold is 6 seconds. A response that takes 6 seconds isn't a failure in the traditional sense (we got data back), but it's slow enough to cause resource pressure if sustained.

The cooling period (how long the circuit stays open before trying half-open) is 30 seconds. Long enough for transient issues to resolve. Short enough that we don't wait unnecessarily when the external service recovers quickly.

The half-open test sends 3 requests to the external service. If 2 of 3 succeed within the normal time threshold, the circuit closes. If any fail, the circuit reopens for another cooling period.

Fallback behavior

When the circuit is open, the traveler still needs a response. "Something went wrong, try again later" is technically accurate but useless. The fallback behavior should be as helpful as possible given the constraint.

For flight search, our fallback is: the AI agent acknowledges that flight search is temporarily unavailable, provides an estimate of when to retry, and offers to help with other aspects of trip planning in the meantime. The agent doesn't pretend flights are available. It doesn't hallucinate results. It clearly communicates the limitation.

For hotel search, similar behavior. The agent explains the limitation and suggests alternatives.

For non-interactive tool calls (like background price monitoring or schedule checking), the circuit breaker failure is handled silently. The job gets requeued with a delay. The traveler never sees the failure because the result wasn't needed immediately.

The fallback is different for each tool because the user impact is different. A failed flight search during an active conversation is highly visible. A failed background check is invisible. The circuit breaker state is the same in both cases, but the user-facing response adapts.

Monitoring circuit breaker state

The circuit breaker state is one of the most important signals on our monitoring dashboard. Each external service has a circuit breaker, and each one displays its current state: closed (green), half-open (yellow), open (red).

State transitions generate alerts. A circuit opening is a significant event that the on-call engineer should know about. A circuit that's been open for more than 5 minutes is a P2 incident. A circuit that opens and closes repeatedly (flapping) indicates an external service that's degraded but not fully down, which is often worse than a clean outage.

We track circuit breaker metrics over time: how often each circuit opens, how long it stays open, how many requests were fast-failed during open state, and what the external service's error rate was when the circuit opened. This historical data helps us tune thresholds and understand the reliability characteristics of each external dependency.

The aggregate view matters too. If multiple circuit breakers open simultaneously, it might indicate a shared dependency failure (like a DNS issue or a network problem) rather than individual service failures. The dashboard shows the correlation.

Recovery detection

The half-open state is the trickiest to get right. Test too aggressively and you overwhelm the recovering service with requests. Test too conservatively and you stay in open state long after the service has recovered.

Our approach is gradual. The half-open state sends a trickle of traffic: 3 test requests over 10 seconds. If these succeed, we don't immediately close the circuit and send full traffic. We increase the trickle to 10% of normal traffic for 30 seconds. If that succeeds, 50% for another 30 seconds. Then full traffic.

This ramped recovery prevents the thundering herd problem. If 50 requests were backed up waiting for the circuit to close, sending all 50 simultaneously could overwhelm the recently-recovered service and cause it to fail again, reopening the circuit. The ramp gives the external service time to absorb the returning load.

Add circuit breakers to your external API integrations

If you depend on external APIs for critical functionality, you need circuit breakers. Not might need. Need. The question isn't whether external APIs have bad days. They all do. The question is whether your system survives those bad days gracefully or cascades into a full outage.

Start with your most critical external dependency. For us, that's the travel data provider. Wrap it in a circuit breaker with conservative thresholds (you can tune later). Add monitoring for state transitions. Define a fallback behavior for the open state. Then extend to other external dependencies.

The implementation is straightforward. The pattern is well-documented. Most languages have libraries that implement it. The hard part isn't the code. It's the fallback behavior: deciding what your application does when the external service is unavailable. That's a product decision disguised as an infrastructure decision, and it deserves as much thought as the circuit breaker configuration itself.

Nowah is an AI travel agent that searches and books real flights and hotels through conversation — no filters, no thirty open tabs. Plan your next trip.