Health Checks for AI Services: Is 'Healthy' Good Enough?

The standard health check for a web service is simple: can the process respond to an HTTP request? If yes, healthy. If no, unhealthy. Restart it.

For an AI travel platform, "the process can respond" is a necessary but laughably insufficient definition of health. Our service might be running, responding to health checks, and returning 200 status codes while the AI model is unreachable, the travel data provider is down, the database is read-only because the disk filled up, and the cache is serving stale data from yesterday.

From the health checker's perspective, everything is green. From the user's perspective, the product is broken. We needed better health checks.

Deep health checks

We implement two health endpoints with different purposes and depths.

The simple liveness probe (`GET /health/live`) answers one question: is the process running and can it handle HTTP requests? This returns 200 if the Node.js event loop is responsive and the HTTP server is accepting connections. It does not check any dependencies. This endpoint is what our container orchestrator uses to decide if a container needs to be restarted.

The deep health check (`GET /health`) is more thorough. It verifies:

Database connectivity. Can we execute a simple query against a relational database? This catches connection pool exhaustion, network partitions, and database server issues.

Cache connectivity. Can we read from and write to an in-memory data store? This catches an in-memory data store outages and connection issues.

Memory system connectivity. Can we reach our agentic memory service? This catches memory system outages that would degrade the agent's ability to use historical context.

Upstream API availability. Can we reach our travel data providers? This does not do a full search, but it verifies network connectivity and authentication.

The deep health check returns a detailed response showing the status of each dependency. If all are healthy, it returns 200. If any critical dependency is down, it returns 503 with details about which dependency failed.

Liveness versus readiness versus startup

These three probe types serve different purposes in a service lifecycle.

Startup probes determine when a newly deployed instance is ready to begin serving traffic. Our startup is not instant: we load configuration, establish database connections, warm the cache, and verify AI model connectivity. The startup probe checks these initialization steps and signals readiness only when all are complete. A cold start takes 5-10 seconds.

Liveness probes determine if a running instance is still functional. If the event loop hangs or the process enters an unrecoverable state, the liveness probe fails and the container restarts. We keep this check lightweight (no external dependency calls) because it runs every few seconds.

Readiness probes determine if a running instance can accept new work. An instance might be alive but not ready if a critical dependency is temporarily unavailable. When readiness fails, the load balancer stops sending new requests to that instance, but the instance stays running. When the dependency recovers, readiness passes again and traffic resumes.

The distinction between liveness and readiness is important for AI services. If the AI model provider has a brief outage, our instances are alive but not ready to serve AI-powered responses. Restarting them (which liveness failure would trigger) accomplishes nothing. Removing them from the load balancer (which readiness failure triggers) is the right response.

Dependency health management

Here is where it gets interesting. When an upstream travel API goes down, is our service "healthy"?

The answer depends on what "healthy" means for the user experience. If the flight search provider is down, we cannot search for flights. But we can still search for hotels, manage existing bookings, continue existing conversations about non-search topics, and provide travel information and tools.

We model this as partial health. The deep health check returns the status of each dependency with a severity level. Critical dependencies (database, cache) failing means the service is unhealthy. Non-critical dependencies (one specific travel provider) failing means the service is degraded but functional.

When a dependency is degraded, we adjust the AI agent's behavior. The agent knows which tools are currently available. If flight search is unavailable, the agent can tell the user "flight search is temporarily unavailable, but I can help with hotels or answer questions about your existing bookings." This is much better than a generic error page.

Automated recovery

When a dependency failure is detected, we want recovery to happen without human intervention whenever possible.

Database connection recovery. If the database connection pool is exhausted, we implement a backoff and retry strategy. New connections are attempted at increasing intervals. If the pool recovers, traffic resumes automatically.

Cache recovery. If an in-memory data store is unreachable, the application falls back to direct database queries. When an in-memory data store becomes available again, the cache warms automatically from subsequent requests. No manual intervention needed.

Upstream API recovery. If a travel provider returns errors, we implement a circuit breaker that stops sending requests for a cooldown period. After the cooldown, we send a probe request. If it succeeds, the circuit closes and normal traffic resumes.

AI model recovery. If the AI model provider is slow or unavailable, we queue user messages and process them when the model returns. For short outages (under 30 seconds), the user sees a "thinking..." indicator. For longer outages, we notify the user that the agent is temporarily unavailable and offer to notify them when it is back.

Each of these recovery paths is tested regularly. We run failure injection tests that deliberately break dependencies and verify that recovery happens correctly. Automated recovery reduces our incident response time for common failures from minutes (human intervention) to seconds (automatic).

Health checks are a small part of the codebase but a disproportionately important part of the operational story. Getting them right means your monitoring reflects reality, your deployments are safe, and your recovery is automatic. Getting them wrong means green dashboards during outages and pager fatigue from false alarms.

Nowah is an AI travel agent that searches and books real flights and hotels through conversation — no filters, no thirty open tabs. Plan your next trip.