Streaming synthesis

The default synthesize returns empty bytes so the server boots without a TTS key. Today we replace it with a real provider. The critical design choice is streaming. A blocking synthesize that returns a finished WAV after a full second feels like the agent went quiet. A streaming version that emits audio as it is generated feels like the agent is talking to you.

agent.py
python
def synthesize(self, text: str) -> bytes:
    """Convert reply text into audio bytes.

    Demo placeholder: returns empty bytes. Wire in a real TTS provider
    (ElevenLabs, Deepgram Aura, Edge-TTS, etc.) when ready.
    """
    if not text:
        return b""
    # Placeholder audio payload, real TTS plugs in here.
    return b""

Same seam pattern as transcribe. The placeholder returns silence so smoke tests pass. A real provider plugs in here with the same method name, which means no other code changes.

agent.py
python
from typing import AsyncIterator

async def synthesize_stream(self, text: str) -> AsyncIterator[bytes]:
    """Stream TTS audio chunks for text.

    Yields audio bytes as they are produced. Consumers can play chunks
    immediately instead of waiting for the full utterance.
    """
    if not text:
        return

    # Pseudocode: replace with your provider SDK (ElevenLabs, Deepgram, Edge-TTS)
    async with tts_provider.stream(text, voice="warm_female_1", format="pcm_16000") as stream:
        async for chunk in stream:
            yield chunk

This is the streaming shape. The method is an async generator that yields chunks as the TTS provider produces them. In the FastRTC handler, every yielded chunk becomes outbound audio immediately.

Quiz: Quiz

Loading practice…