User interrupts LLM

Real humans talk over each other. A voice agent that keeps speaking while the caller interrupts feels rude and broken. Barge-in is the behavior where the moment the caller speaks, the agent stops and listens. It is not optional for a production phone agent.

router.py

python

import asyncio

_current_reply_task: asyncio.Task | None = None

async def _audio_handler(audio_chunk):
    global _current_reply_task

    # If the caller is speaking, cancel the agent's reply immediately
    if _current_reply_task and not _current_reply_task.done():
        _current_reply_task.cancel()

    audio_bytes = bytes(audio_chunk) if not isinstance(audio_chunk, (bytes, bytearray)) else audio_chunk
    transcript = _agent.transcribe(audio_bytes)

    async def _run_turn():
        reply = await _agent.respond(transcript)
        async for chunk in _agent.synthesize_stream(reply):
            yield chunk

    # Track the new reply so the next incoming audio can cancel it
    _current_reply_task = asyncio.create_task(_drain(_run_turn()))
    async for chunk in _run_turn():
        yield chunk

The core trick is holding a reference to the current reply task and cancelling it the next time the VAD-gated handler fires. FastRTC already runs the handler on caller speech, so caller speech becomes the cancel signal.

Barge-in flow

The caller interrupts, the agent stops, and the new turn begins.

You could, but it wastes tokens and risks the LLM finishing a turn the caller no longer wants. Cancelling the LLM stops the generation at its current token. The TTS stream is attached to that same task, so cancelling the parent cancels the whole pipeline. Clean cancellation upstream is always better than trying to mute a pipeline you already paid for.

Quiz: Quiz

Loading practice…