Graceful dependency checks

Readiness should return true only when the service can actually handle a request. That usually means the LLM client is initialised, an API key is present, and anything else a handler needs to work. The probe must also be fast, because the orchestrator calls it every few seconds.

router.py

python

import os
from fastapi import HTTPException
from service import llm_provider

@router.get("/health/ready")
async def readiness():
    """
    Readiness: are all dependencies wired up so we can serve traffic?
    Kubernetes pulls the pod out of the Service's endpoint list while
    readiness is failing, without restarting it.
    """
    checks = {
        "llm_provider": llm_provider is not None,
        "llm_api_key_set": bool(
            os.getenv("OPENROUTER_API_KEY")
            or os.getenv("FIREWORKS_API_KEY")
            or os.getenv("GEMINI_API_KEY")
            or os.getenv("OPENAI_API_KEY")
        ),
    }
    ready = all(checks.values())
    if not ready:
        raise HTTPException(status_code=503, detail={"status": "not_ready", "checks": checks})
    return {"status": "ready", "checks": checks}

Two cheap checks: the provider object exists, and at least one API key is set. A failed readiness returns 503 with a structured detail so the operator can see which check failed without digging through logs.

Flashcards: Flashcards

Loading practice…

Checkpoint: Health probes checkpoint

Loading practice…