The text/event-stream contract

The generator is the content. The contract is the envelope. SSE has specific requirements: a media type, a cache policy, and, when you run behind nginx or a cloud load balancer, a header that disables buffering. Miss one of these and the browser either never sees events or sees them all at the end.

router.py
python
from fastapi import APIRouter
from fastapi.responses import StreamingResponse

from models import ChatRequest
from service import generate_chat_stream

router = APIRouter(prefix="/deploy-patterns", tags=["deploy-patterns"])

@router.post("/chat/stream")
async def chat_stream(request: ChatRequest):
    return StreamingResponse(
        generate_chat_stream(request),
        media_type="text/event-stream",
        headers={
            "Cache-Control": "no-cache",
            "Connection": "keep-alive",
            "X-Accel-Buffering": "no",
        },
    )

text/event-stream tells the browser to parse chunks as SSE. Cache-Control no-cache tells proxies not to store the stream. X-Accel-Buffering no is an nginx hint that disables response buffering so chunks flow through in real time.

Proxies and load balancers kill idle connections. A long LLM pause can look idle. SSE has a built-in answer: comment lines that start with a colon. They are valid on the wire, the browser ignores them, and they reset the proxy timer.

service.py
python
async def _heartbeat_ticker() -> None:
    try:
        while True:
            await asyncio.sleep(15)
    except asyncio.CancelledError:
        return

A background task that ticks every 15 seconds. In a production setup you would interleave a yield like ": heartbeat\n\n" inside the stream on each tick. The cancellation path is what keeps the task clean when the request finishes.

Quiz: Quiz

Loading practice…