The basic voice agent

A voice agent is a short loop of specialised parts. Speech to text turns your voice into words. An LLM reads those words and writes a reply. Text to speech turns the reply into audio. Voice activity detection decides when you are done speaking so the agent knows when to respond.

restaurant_agent.py
python
from livekit.agents import JobContext, WorkerOptions, cli
from livekit.agents.voice import Agent, AgentSession
from livekit.plugins import silero, deepgram
from utils.livekit_utils import get_livekit_llm

class RestaurantAgent(Agent):
    def __init__(self) -> None:
        super().__init__(
            instructions="You are a friendly restaurant assistant. Keep replies short and conversational.",
            stt=deepgram.STTv2(model="flux-general-en", eager_eot_threshold=0.3),
            llm=get_livekit_llm(),
            tts=deepgram.TTS(model="aura-asteria-en"),
            vad=silero.VAD.load(),
        )

async def entrypoint(ctx: JobContext):
    await ctx.connect()
    session = AgentSession()
    await session.start(agent=RestaurantAgent(), room=ctx.room)

if __name__ == "__main__":
    cli.run_app(WorkerOptions(entrypoint_fnc=entrypoint, agent_name="restaurant-agent"))

This is a complete voice agent in under twenty lines. STT uses Deepgram flux for low latency transcription. TTS uses aura-asteria for a natural voice. Silero VAD runs locally and detects when you finish speaking.

The setting worth paying attention to is eager_eot_threshold. End of turn detection decides how quickly the agent jumps in after you stop talking. A value of 0.3 means the agent waits 300 milliseconds of silence before replying. Too low and it interrupts. Too high and it feels sluggish. That single number controls how alive the conversation feels.

terminal
bash
# Start the agent worker
uv run python restaurant_agent.py dev

# In another terminal, start the FastAPI backend
uv run uvicorn main:app --reload

# Now open the test playground
open https://agents-playground.livekit.io

The agent runs as a worker and waits for room dispatches. The playground is a hosted LiveKit UI that lets you join a room and talk to the agent before the frontend is built.

AI prompt: Try it: tune a voice prompt

Loading practice…

Quiz: Quiz

Loading practice…

Checkpoint: Voice fundamentals checkpoint

Loading practice…