Closing SSE streams on shutdown

Background jobs are not the only in-flight work on shutdown. Long-running SSE generators are also alive when SIGTERM hits. If you do nothing, the connection drops mid-token and the client sees a TCP reset. The fix is catching the cancellation, sending a terminal event, and exiting cleanly.

router.py

python

async def chat_stream():
    try:
        async for token in llm_provider.stream(prompt):
            yield f"data: {json.dumps({'content': token})}\n\n"
        yield "data: {\"done\": true}\n\n"
    except asyncio.CancelledError:
        # SIGTERM landed mid-stream. Tell the client we are stopping.
        yield "data: {\"done\": true, \"reason\": \"shutdown\"}\n\n"
        raise

Wrap the generator body in try and except CancelledError. Yield a final event so the browser SSE client sees end-of-stream, then re-raise so uvicorn knows the stream is closed.

Quiz: Quiz

Loading practice…