Pre-audio loop

Adding audio to a broken LLM loop is how you waste an afternoon. Before we touch WebRTC, we exercise the same agent, with the same system prompt, using text in and text out. If the reply is short, warm, and free of markdown, the audio loop has a real chance. If it is not, audio will not save us.

router.py
python
@router.post("/turn", response_model=TranscribeResponse)
async def run_turn(req: TranscribeRequest):
    """Run one text-in / text-out agent turn.

    Useful for testing the LLM path without a WebRTC client.
    """
    reply = await _agent.respond(req.text)
    return TranscribeResponse(reply=reply, caller_name=req.caller_name or "Caller")

A plain POST endpoint that exercises VoiceAgent.respond. Same system prompt, same provider, same length constraints. If this endpoint produces a bullet list, the audio version will too, and it will sound awful.

agent.py
python
async def respond(self, user_text: str) -> str:
    """Run user_text through the LLM with the phone-agent system prompt.
    Returns a short spoken-style reply.
    """
    try:
        from utils.llm_provider import get_llm_provider
        provider = get_llm_provider()
        prompt = f"{self.system_prompt}\n\nCaller said: {user_text}\n\nYour short spoken reply:"
        reply = await provider.generate_text(prompt, max_tokens=120, temperature=0.5)
        return (reply or "").strip()
    except Exception as e:
        logger.warning(f"LLM provider unavailable, using canned reply: {e}")
        return "Thanks for calling SkyHop. Our system is briefly offline, please try again shortly."

Two details to notice. max_tokens is capped at 120 because spoken replies should be short. The except branch returns a safe canned reply, because silence on a phone call is worse than a graceful fallback sentence.

terminal
bash
curl -s -X POST http://localhost:8000/phone/turn \
  -H 'Content-Type: application/json' \
  -d '{"text":"Hi, I want to change my seat on flight SH101","caller_name":"Alex"}' | jq

# Expected: a short, warm reply with no markdown, no asterisks, no bullets

Treat this as a smoke test. You should see a one sentence reply, with contractions, under roughly twenty spoken words. If the model returns a list, you have a prompt problem to fix before audio enters the picture.

Because audio hides every LLM problem inside a cloud of codec noise, STT errors, and latency. When the agent sounds bad, you cannot tell if the prompt is wrong, the transcription missed a word, or the TTS chopped a syllable. A text baseline isolates the LLM behavior so that when audio is strange later, you already trust this layer.

Quiz: Quiz

Loading practice…