Latency budget math
Humans feel conversation as broken above roughly 800 milliseconds of silence after they stop talking. That is the entire budget we have to split between STT finalization, LLM first token, TTS first audio, and network round trips. If you do not know where those milliseconds go, you cannot design the loop.
Voice turn latency budget
Each leg of a single voice turn and the time it consumes.
A workable budget for a warm voice agent: VAD and STT finalization around 150 ms, LLM first token around 300 ms, TTS first audio around 200 ms, network and frame jitter around 150 ms. That sums to roughly 800 ms. Any leg that blows its budget blows the whole turn.
Matching exercise: Match each leg to the technique that keeps it fast
Loading practice…
Ordering exercise: Order a single voice turn from caller pause to caller hears reply
Loading practice…
Quiz: Quiz
Loading practice…