Chunked playback

Streaming synthesis only helps if the handler yields chunks the moment they arrive. A naive handler that awaits the full audio and then yields it once collapses streaming back into batch. The shape of the handler is where streaming is won or lost.

router.py
python
async def _audio_handler(audio_chunk):
    audio_bytes = bytes(audio_chunk) if not isinstance(audio_chunk, (bytes, bytearray)) else audio_chunk

    transcript = _agent.transcribe(audio_bytes)
    reply_text = await _agent.respond(transcript)

    # Stream TTS chunks so the caller hears the first syllable fast
    async for audio_chunk_out in _agent.synthesize_stream(reply_text):
        yield audio_chunk_out

Notice what changed. synthesize_stream is an async generator, and the handler simply yields each chunk as it arrives. Nothing is buffered. The first chunk hits the caller while the rest of the reply is still being synthesized.

Batch vs chunked playback

Same TTS work, very different perceived latency.

In a single streaming session, no. The TTS provider emits chunks in order and your async generator yields them in the same order. WebRTC transports audio with sequence numbers so even if packets reorder in flight, the receiver reconstructs them in order. The worst case is a dropped packet, which produces a small glitch, not garbled speech.

Quiz: Quiz

Loading practice…