Graceful fallback when the LLM fails

The LLM step is optional. If the cloud provider is rate-limited or the local server is offline, the app must still work. The backend returns the raw transcript on LLM failure, and the frontend shows it immediately while cleanup happens in the background.

backend/transcription.py
python
def clean_with_llm(self, text, system_prompt=None):
    if not text:
        return ''

    prompt_to_use = system_prompt if system_prompt else SYSTEM_PROMPT

    try:
        response = self.llm_client.chat.completions.create(
            model=self.llm_model,
            messages=[
                {'role': 'system', 'content': prompt_to_use},
                {'role': 'user', 'content': text},
            ],
            temperature=0.3,
            max_tokens=200,
        )
        cleaned = response.choices[0].message.content.strip()
        return cleaned
    except Exception as e:
        print(f'LLM error: {e}')
        return text  # Fallback to raw text

Returning text on exception means the pipeline degrades gracefully. Users see the raw Whisper output instead of a broken UI. Log the error so you can fix the root cause later.

frontend/src/App.tsx
typescript
const transcribeData = await transcribeResponse.json();
setRawText(transcribeData.text || '');
setIsProcessing(false);

if (useLLM && transcribeData.text) {
  setIsCleaningWithLLM(true);
  const cleanResponse = await fetch('/api/clean', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ text: transcribeData.text, system_prompt: systemPrompt }),
  });
  // ... update cleaned text when it arrives
  setIsCleaningWithLLM(false);
}

The frontend shows raw text the moment transcription finishes. The cleanup call runs after, so users never stare at a blank screen waiting for the LLM.

Log it for yourself, hide it from the user. The raw transcript is a legitimate result and showing an error banner for a fallback path trains users to ignore errors. Reserve visible errors for cases where the user can act, like microphone permission denial.

Quiz: Quiz

Loading practice…

Checkpoint: LLM cleanup checkpoint

Loading practice…