--- title: Building Real-Time AI Systems With unidirectional server streaming description: "bidirectional sockets vs server streaming? For streaming AI responses, server streaming wins on simplicity, reliability, and HTTP compatibility. Here is the technical deep dive." canonical: https://nowah.xyz/blog/building-realtime-ai-sse lastModified: "2026-08-07T08:05:01.235Z" --- # Building Real-Time AI Systems With unidirectional server streaming bidirectional sockets vs server streaming? For streaming AI responses, server streaming wins on simplicity, reliability, and HTTP compatibility. Here is the technical deep dive. The moment we decided that our AI travel agent would stream responses in real time, we faced a fundamental protocol decision: bidirectional sockets or unidirectional server streaming? We chose server streaming. Eighteen months later, I am confident it was the right choice. Here is why, and how we built the streaming architecture that makes our agent feel alive. ## Why real-time matters for AI ![Illustration for this section](https://pics.nowah.xyz/website-media/ai-research-063-img-1.webp) The loading spinner is the enemy of conversational AI. When a user asks "find me flights to Tokyo" and sees a spinning circle for five seconds, the experience breaks. They are not using a search engine where waiting is expected. They are having a conversation, and conversations do not have five-second pauses with no indication of what is happening. Real-[time streaming](/blog/lessons-building-real-time-streaming-ai) transforms that five-second wait into an engaging experience. The user sees the agent thinking, searching, processing, and composing, all in real time. The same five seconds feels productive rather than dead. First-token streaming latency under 500 milliseconds for frontier models means the user sees activity almost immediately after sending a message. This single number, time to first visible response, is the [most important](/blog/why-speed-is-most-important-feature) latency metric for conversational AI products. ## bidirectional sockets vs. server streaming bidirectional sockets provide bidirectional, full-duplex communication. Either side can send data at any time. This is perfect for applications like collaborative editing or multiplayer games where both parties are constantly exchanging data. server streaming provides unidirectional, server-to-client communication over a standard HTTP connection. The server pushes events to the client. The client sends messages through normal HTTP requests. For AI response streaming, the communication pattern is inherently unidirectional: the server generates a response and pushes it to the client progressively. The client does not need to send data to the server during the streaming phase. They already sent their message via an HTTP POST. They just need to receive the response as it generates. This makes server streaming a natural fit. We get all the streaming capability we need with significantly less complexity: **HTTP compatibility.** server streaming runs over standard HTTP. It works through proxies, load balancers, CDNs, and firewalls without special configuration. bidirectional sockets require protocol upgrades that some infrastructure does not handle gracefully. **Automatic reconnection.** The server streaming specification includes built-in reconnection with last-event-ID tracking. If the connection drops (common on mobile networks), the browser automatically reconnects and can resume from where it left off. bidirectional sockets require manual reconnection logic. **Simpler server implementation.** An server streaming endpoint is a normal HTTP handler that keeps the connection open and writes events. No connection upgrade, no frame parsing, no ping/pong heartbeats. **Lower resource usage.** server streaming connections are lighter than a bidirectional socket connections because they do not maintain a bidirectional channel. At scale, this translates to more concurrent connections per server instance. The main limitation of server streaming is that it is server-to-client only. For features that require bidirectional real-time communication (like typing indicators from the user side, or presence tracking), we use separate mechanisms. But for the core use case of streaming AI responses, server streaming is simpler, more reliable, and more compatible with standard infrastructure. ## Event schema design ![Supporting diagram](https://pics.nowah.xyz/website-media/ai-research-063-img-2.webp) The key architectural decision is the event schema: what types of events does the server send, and what data does each carry? We defined five event types: **onStatus**: operational updates about what the agent is doing. "Searching flights..." "Checking hotel availability..." These render as thinking indicators in the UI. Payload: a short status string. **onThinking**: the agent's internal reasoning, if we choose to display it. "The user prefers direct flights, so I am filtering for non-stop options." Payload: a reasoning text chunk. **onChunk**: a text token in the agent's response. These arrive rapidly and compose the visible response as they stream. Payload: a text string (often just a few words or a sentence fragment). **onToolResult**: structured data from a tool call. A flight search result, a hotel recommendation, a weather forecast. Payload: a typed JSON object with the tool name and structured result data. This is what triggers flight cards, hotel cards, and other rich components to render in the conversation. **onComplete**: the final event, signaling that the response is finished. Payload: the complete message content, final session state, and any offers or cards that should be persisted. Each event is typed and parseable. The client knows exactly what to do with each event type. onChunk events append to the response text. onToolResult events render a card component. onStatus events update the [thinking indicator](/blog/thinking-indicator-most-important-animation). onComplete triggers state cleanup and message persistence. ## Client-side state management Building a coherent UI from a stream of events requires careful [state management](/blog/state-management-ai-conversations). The client maintains a response accumulator that builds up over the course of the stream. As onChunk events arrive, they are appended to the response text. As onToolResult events arrive, they are added to a tool results array. The UI renders from this accumulator, updating progressively as new events arrive. The tricky part is handling interleaving. The agent might start generating text, then call a tool (triggering a status event and a tool result event), then continue generating text that references the tool result. The accumulator needs to maintain the correct ordering so the UI renders text, then a card, then more text about the card. We handle this with a sequential event buffer. Events are processed in order. If a toolResult arrives between text chunks, it is inserted at the correct position in the response. The final rendered message has text and cards interleaved naturally, matching the order the agent generated them. ## Error recovery Mobile networks are unreliable. Connections drop. Tunnels kill signals. Network switches introduce latency spikes. An server streaming connection that drops mid-stream needs to recover gracefully. Our error recovery strategy: **Reconnection with event replay.** Each event includes a sequence number. When the client reconnects, it sends the last received sequence number. The server replays any events after that number. The user sees a brief interruption, then the stream continues from where it stopped. **Timeout detection.** If no event arrives for 15 seconds during an active stream, the client assumes the connection is lost and initiates reconnection. A heartbeat event (an empty comment in server streaming) is sent every 10 seconds to keep the connection alive through infrastructure that might close idle connections. **\[Graceful degradation\]\(/blog/graceful\-degradation\-slow\-ai\)\.** If reconnection fails after three attempts, the client falls back to a polling mechanism: request the complete response via a normal HTTP call. The streaming experience is lost, but the response is still delivered. Mobile networks add 100-2000ms of variable latency. Our error recovery is tuned for this: aggressive enough to detect real failures quickly, patient enough to tolerate momentary latency spikes without false positives. ## Performance at scale Each active server streaming connection consumes a server resource (a held-open HTTP connection). At scale with thousands of concurrent conversations, connection management matters. We run our server streaming endpoints on infrastructure configured for high connection counts. Each server instance handles thousands of concurrent server streaming connections. [Connection pooling](/blog/connection-pooling-under-pressure) and efficient event dispatch keep memory usage per connection low. Edge deployment helps with latency. server streaming connections terminate at the edge server nearest to the user, and the edge proxies events from the origin. This reduces the round-trip time for event delivery, especially for mobile users on cellular networks. ## Lessons from production **Streaming everything was wrong.** Early on, we streamed every character of every response. This created too many events and visible "jitter" as individual words appeared. We switched to streaming in small chunks (phrases or short sentences) which reads more naturally. **Client rendering speed matters.** If events arrive faster than the client can render them, the UI falls behind. We implemented a render queue that batches events when they arrive faster than the 60fps render cycle. **Debug tooling is worth the investment.** We built a stream inspector that logs every event with timing, making it easy to diagnose latency issues, event ordering problems, and rendering bugs. Without it, debugging streaming interactions is painful. server streaming is the right choice for streaming AI responses in production. It is simpler than bidirectional sockets, more compatible with standard infrastructure, and provides everything a unidirectional streaming use case needs. The engineering effort is in the event schema, the client-side state management, and the error recovery, not in the protocol itself. --- Nowah is an AI travel agent that searches and books real flights and hotels through conversation — no filters, no thirty open tabs. [Plan your next trip](https://app.nowah.xyz).