Streaming

Streaming is the technique of receiving language model outputs incrementally as they're generated, rather than waiting for the complete response. This provides a better user experience by showing content immediately and giving the perception of faster responses, even when total generation time remains the same.

Technically, streaming involves receiving server-sent events (SSE) or WebSocket messages containing individual tokens or small chunks as the model produces them. Most LLM APIs (OpenAI, Anthropic, etc.) support streaming endpoints that enable this progressive rendering.

Implementing streaming requires: handling partial JSON or text chunks, managing response state as data arrives, implementing proper error handling for interrupted streams, and designing UIs that gracefully display incomplete content. Streaming is standard practice for any user-facing LLM application and significantly impacts perceived performance and user satisfaction.