--- title: "User Research for AI Products: What We Learned" description: "We test conversations, not click paths. User research for AI products means watching how people talk to the AI, where trust breaks, and when they switch from text to voice." canonical: https://nowah.xyz/blog/user-research-ai-products-what-learned lastModified: "2026-08-07T08:03:28.697Z" --- # User Research for AI Products: What We Learned We test conversations, not click paths. User research for AI products means watching how people talk to the AI, where trust breaks, and when they switch from text to voice. Traditional user research for apps follows a well-established playbook. Give users a task. Watch where they tap. Observe where they get stuck. Count the clicks to completion. Analyze the funnel drop-offs. This playbook does not work for conversational AI products. There is no predetermined click path. There is no funnel. Two users with the same goal might have completely different conversations, ask for different things at different times, and arrive at the same booking through different routes. The research method needs to match the product. We test conversations, not click paths. Here is what we learned. ## Give users a scenario, watch how they converse ![Illustration for this section](https://pics.nowah.xyz/website-media/design-053-img-1.webp) Our research sessions start with a travel scenario: "You want to plan a long weekend trip to Barcelona with your partner in April. Budget is around $2,000 total." Then we watch. We do not tell users how to interact. We do not show them the interface first. We hand them the phone with the chat open and observe. The first observation is always about phrasing. Some users type complete sentences: "I would like to find flights to Barcelona from New York departing April 10th." Others type fragments: "Barcelona April flights." Others tap the voice button and speak naturally: "Hey, I want to go to Barcelona with my girlfriend, maybe second week of April, we have about two thousand to spend." Each phrasing style teaches us something about how the AI needs to parse and respond. The complete sentences are easy. The fragments need the AI to ask clarifying questions without being annoying. The voice messages need natural language understanding that handles conversational filler and imprecise language. ## Trust breakpoints: the moments users hesitate The most valuable data from user research is where trust breaks. These are moments when the user pauses, re-reads something, looks uncertain, or says something like "wait, is this right?" Each hesitation is a design signal. Common trust breakpoints in travel AI include price presentation without context. Users who see "$450" on a flight card without understanding why the AI chose that flight hesitate more than users who see "$450 — this is the only nonstop under $500 that departs after 10 AM." The reasoning eliminates the hesitation. Another breakpoint is the transition from browsing to buying. The moment the user needs to tap "Book" and commit money, trust becomes critical. Users who saw the AI explain its reasoning throughout the conversation were significantly more likely to accept the first recommended option compared to users who only saw options without explanations. Tool-use status is another breakpoint. When the AI says "Searching flights..." users who see a visible indicator with elapsed time maintain confidence. Users who see nothing for several seconds start doubting whether the app is working. ## A/B testing message formats ![Supporting diagram](https://pics.nowah.xyz/website-media/design-053-img-2.webp) We run A/B tests on how the AI presents information, not just what it presents. One variant shows three flight options silently: just the cards, no commentary. Another variant prefaces the cards with a sentence explaining the curation: "I found [three options](/blog/why-three-options-not-three-hundred) — the first is cheapest, the second is fastest, the third is the best balance of both." The variant with reasoning consistently outperforms. Users engage with all three cards more frequently. They make faster decisions. They report higher confidence in their choice. The reasoning does not add much information — the user could compare the cards themselves — but it adds context that frames the decision. We also test streaming versus block delivery. Streaming shows text token by token as the AI generates it. Block delivery waits for the full response and shows it all at once. Users perceive [streaming responses](/blog/streaming-ai-responses-real-time-chat) as a large share faster, even when the actual delivery time is identical. The progressive reveal makes the wait feel shorter because the user is reading instead of staring at a spinner. ## Voice versus text switching patterns One of the more interesting findings from our research is when and why users switch between text and voice input. The pattern is surprisingly consistent across user segments. Voice peaks in the evening and nighttime hours. Users lying in bed, relaxing on the couch, or winding down after work prefer speaking to typing. Voice also peaks during commutes and other moments where hands are occupied. Text dominates during work hours and in public settings. Users at their desk prefer typing because it is quieter and more private. Users on a bus or in a cafe prefer typing because speaking to a phone feels conspicuous. The switch from text to voice often happens at frustration points. A user who has typed three messages trying to describe a complex requirement switches to voice because speaking is faster for nuanced requests. "I want a flight that arrives before noon local time but I do not want to leave before 8 AM and I prefer an aisle seat and nonstop if possible" is exhausting to type and effortless to say. This finding directly influenced our design. The microphone button is always visible and equally prominent as the text input. The transition between input modes is seamless. Voice input appears as transcribed text in the chat, maintaining the same conversation format regardless of input method. ## Measuring trust Trust is the meta-metric for an AI product. We measure it through proxy behaviors: acceptance rate of first recommendations, time spent on the review screen before confirming, and post-booking satisfaction scores. Users who complete the full onboarding and provide their preferences show significantly higher satisfaction rates than users who skip onboarding. This validates the [preference collection](/blog/preference-collection-as-onboarding) design: the investment in answering five questions pays off in perceived AI quality for every subsequent interaction. The most reliable trust indicator is what we call "follow-up specificity." Trusting users give the AI more detailed follow-up requests: "Actually, can you check if there is a hotel closer to La Rambla with a rooftop pool?" Distrusting users give generic responses: "No, show me something else." The specificity indicates the user believes the AI will understand and act on their nuanced request. ## Setting up a conversation-based research protocol If you are researching an AI product, start with scenarios, not tasks. Give users a goal, not a path. Record the conversation transcript and annotate it for trust breakpoints, input mode switches, and phrasing patterns. A/B test not just features but presentation formats. And measure [trust through](/blog/building-trust-through-transparency-security) behavioral proxies, not just satisfaction surveys. The conversation is your product, and your research should study the conversation. --- Nowah is an AI travel agent that searches and books real flights and hotels through conversation — no filters, no thirty open tabs. [Plan your next trip](https://app.nowah.xyz).