Why We Build Our Own AI Evaluation Framework
How do you test a product whose output is different every time? Standard testing fails for AI agents. Here is our eval framework and why it matters.
Standard software testing assumes deterministic behavior: input A always produces output B. If it does not, the test fails. Something is broken.
AI does not work this way. Input A might produce output B, C, or D, all of which could be valid. The same flight search query can return different recommended options on different runs, depending on the model's reasoning path, the current inventory, and subtle variations in context processing.
This breaks every traditional testing assumption. So we built our own evaluation framework.
Why standard tests fail
A unit test that asserts "the agent recommends United flight 847 for this query" will fail randomly because the agent might correctly recommend a different flight that is equally good or even better. The test is wrong, not the agent.
Integration tests that expect specific API call sequences fail because the agent might choose a different, equally valid approach to satisfying the same request. The test is too rigid for a non-deterministic system.
End-to-end tests that rely on exact output matching are worthless because the outputs are never exact.
Our approach
Our evaluation framework measures quality within acceptable ranges rather than asserting exact matches.
Accuracy evaluations. Does the agent correctly interpret the request? Does it search the right routes? Does it apply stated constraints? These are evaluated against criteria, not exact outputs. "Did the agent search for flights to Tokyo in April?" not "Did the agent return exactly these three flights?"
Relevance evaluations. Are the recommendations relevant to the user's stated needs? If someone asks for direct flights under five hundred dollars, does the recommendation include direct flights under five hundred dollars? This is a constraint check, not an exact match.
Safety evaluations. Does the agent refuse to do things it should not? Does it handle edge cases without producing dangerous output? Safety evals are the strictest category because failures here have the highest consequences.
Latency evaluations. Does the agent respond within acceptable time bounds? Our framework uses 60-second timeouts per test, reflecting real-world latency requirements. Tests run sequentially to avoid rate limiting from our data providers.
The growing test suite
Every user-reported error becomes a new test case. This means the evaluation suite grows continuously. Errors that are caught, fixed, and added to the test suite can never regress undetected.
The suite now includes hundreds of travel-specific scenarios: simple one-way flights, complex multi-leg itineraries, budget constraints, preference matching, ambiguous requests, and deliberately adversarial inputs.
Each test allows two retries to account for the inherent non-determinism of AI responses. A test that passes on retry is not a failure. It is a valid variation.
Human evaluation alongside automation
Automated evals catch measurable issues. But some quality dimensions resist automation. Is the agent's tone appropriate? Are the explanations clear? Does the conversation feel natural? These require periodic human review alongside automated scoring.
We conduct regular manual reviews of randomly sampled conversations, scoring them on dimensions that automated tests cannot capture. These reviews surface issues that metrics miss.
For AI builders
If you are building an AI product, start your evaluation framework on day one. Not day 100. Every day without a robust eval framework is a day you are shipping changes without knowing whether they improved or degraded the product.
The framework does not need to be perfect at the start. It needs to exist and grow. Every error, every edge case, every user complaint becomes a test case. The suite compounds in value. A year from now, you will have an evaluation framework that gives you confidence to ship quickly because you know, empirically, whether each change makes things better.
Nowah is an AI travel agent that searches and books real flights and hotels through conversation — no filters, no thirty open tabs. Plan your next trip.