Voice activity detection

Without VAD, the agent either waits too long and feels slow, or it jumps in early and talks over the caller. Voice activity detection answers one question over and over: is someone speaking right now? FastRTC gives you ReplyOnPause, which wraps your handler and only runs it when VAD reports the caller has finished.

router.py

python

from fastrtc import Stream, ReplyOnPause

async def _audio_handler(audio_chunk):
    audio_bytes = bytes(audio_chunk) if not isinstance(audio_chunk, (bytes, bytearray)) else audio_chunk
    transcript, reply, audio_out = await _agent.handle_turn(audio_bytes)
    yield audio_out

# ReplyOnPause gates the handler on VAD pause detection
stream = Stream(
    handler=ReplyOnPause(_audio_handler),
    modality="audio",
    mode="send-receive",
)

ReplyOnPause wraps the raw handler. FastRTC runs its VAD continuously on incoming audio and only invokes the handler once the caller has paused for a configured window. Without this, you would invoke the agent on every frame, which is useless.

VAD-gated turn detection

ReplyOnPause keeps the handler quiet until the caller actually finishes.

Quiz: Quiz

Loading practice…