Streaming makes an AI interface feel responsive because the user can see progress before the full answer is ready. It also turns one request into a lifecycle that crosses the model provider, application server, reverse proxy, and browser.
I want that lifecycle to have defined states. A connection closing, a provider finishing, and an answer passing validation are different events. Treating all three as “done” makes failures difficult to explain.
Choose a transport that matches the interaction
Server-sent events work well for a stream of server-to-client updates. The event format supports named events, IDs, and data fields, as described in MDN’s SSE guide.
The browser’s native EventSource interface is convenient for receiving a stream, but applications requiring a POST body or custom request headers may instead use a fetch-based streaming client. That choice changes which parsing and reconnection behavior the application must implement itself.
I make authentication, request submission, and stream subscription explicit. Credentials should not be placed in a URL merely because it is convenient for opening a connection.
Define the application protocol
A useful stream might distinguish acceptance, progress, text, validation status, completion, and failure. Event names are application choices; they are not guarantees supplied by the transport.
event: accepted
data: {"request_id":"req_example"}
event: text_delta
data: {"text":"The available sizes are"}
event: completed
data: {"request_id":"req_example","status":"complete"}This example illustrates framing. A real parser must handle network chunks that split an event, contain several events, or split a multi-byte character. A network read is not automatically one complete message.
Bound the amount of data waiting in memory
If the downstream connection is slow, the server can receive provider output faster than it can deliver it. Unbounded buffering turns a slow client into a memory problem.
The application should respect writable-stream pressure and define a buffer limit. Where the upstream protocol cannot be paused usefully, cancellation may be safer than allowing the queue to grow indefinitely.
MDN’s Streams API concepts explains backpressure as a flow-control mechanism. In an LLM application, I still need to connect that mechanism to provider cancellation and the product’s timeout policy.
Propagate cancellation through the entire request
Closing the browser connection should trigger cleanup in the application and, where supported, cancellation of the upstream model request. Timers, listeners, and buffers need to be released even if cancellation itself fails.
However, the client disconnecting does not prove that the provider stopped billable work. Usage accounting should preserve that uncertainty and reconcile it according to the provider’s available usage information.
Long-lived connections also pass through proxies with buffering and timeout settings. I test the deployed path because a stream that works directly against a development server can behave differently behind production infrastructure.
Decide how partial answers are presented
An answer that fails halfway through should remain visibly incomplete. Reconnecting must not silently create another model call or append a fresh answer to the partial one.
Resumption requires a retained event history and a defined cursor; an event ID alone does not implement replay. For a simpler product, offering an explicit restart may be the clearer behavior.
Validation creates another choice. If factual checks must finish before delivery, the relevant content needs to be buffered. Showing text immediately and checking it afterward is a different product promise.
I verify slow clients, aborted requests, proxy timeouts, provider failures, and malformed event boundaries. A good streaming interface feels quick while still making its completion state trustworthy.
Updated 25 September 2026.