Guardrails on misclassification

Guardrails are not a single feature, they are a layered strategy. The prompt constrains the model. The JSON parser handles malformed output. The membership check validates the label. The downstream node validates its own input. Each layer catches a different failure mode.

booking_graph.py
python
async def _llm_json(prompt: str, system: str) -> Dict[str, Any]:
    """Call the LLM and parse a JSON object out of the response."""
    provider = get_llm_provider()
    full_prompt = f"[SYSTEM]\n{system}\n\n[USER]\n{prompt}"
    raw = await provider.generate_text(full_prompt, temperature=0.2, max_tokens=400)
    text = (raw or "").strip()

    # Layer 1: strip code fences
    if text.startswith("```"):
        text = text.strip("`")
        if text.lower().startswith("json"):
            text = text[4:]
        text = text.strip()

    # Layer 2: try strict parse
    try:
        return json.loads(text)
    except json.JSONDecodeError:
        # Layer 3: recover the first {...} block
        start = text.find("{")
        end = text.rfind("}")
        if start != -1 and end != -1 and end > start:
            try:
                return json.loads(text[start : end + 1])
            except json.JSONDecodeError:
                pass
        logger.warning("LLM JSON parse failed, raw=%r", raw[:200])
        return {}  # Layer 4: return empty dict, caller handles default

The JSON parser has its own guardrails. Code fences, partial JSON, missing response, all handled. When every layer fails, the caller gets an empty dict and can fall back to sane defaults.

booking_graph.py
python
async def find_doctor(state: BookingState) -> BookingState:
    specialty = state.get("specialty") or "general_medicine"
    doctors = list_doctors_by_specialty(specialty)
    if not doctors:
        # If the specialty exists in our vocab but no doctors match,
        # degrade to general_medicine rather than returning None.
        doctors = list_doctors_by_specialty("general_medicine")
    state["doctor"] = doctors[0] if doctors else None
    _record_event(
        state,
        "find_doctor",
        {"doctor": state["doctor"]["name"] if state.get("doctor") else None},
    )
    return state

Even though the classifier already validated against the vocabulary, the downstream node defends itself. Empty result set falls back to general medicine. This defense-in-depth keeps the graph running when any single layer misbehaves.

Flashcards: Flashcards

Loading practice…

Quiz: Quiz

Loading practice…