Validation patterns

Validation is simple: define what is acceptable, then check the LLM output against those rules before executing. If invalid, return a specific error message.

You should never rely on the LLM to validate itself. Validation must be deterministic: plain Python code that checks exact rules. The LLM generated the bad output in the first place, so trusting it to catch its own errors is unreliable.

07-error-handling.ipynb
python
SUPPORTED_ROOMS = ["kitchen", "bedroom", "living room"]

def validate_command(data):
    """Validate a smart home command.
    Returns None if valid, error string if invalid."""

    # Check room exists
    if data.get("room") not in SUPPORTED_ROOMS:
        return (
            f"Room '{data.get('room')}' is not supported. "
            f"Choose from {SUPPORTED_ROOMS}."
        )

    # Check temperature is numeric
    if not isinstance(data.get("temp"), (int, float)):
        return "Temperature must be a number."

    # Check temperature range
    temp = data.get("temp")
    if temp < 15 or temp > 30:
        return "Temperature must be between 15 and 30."

    return None  # Valid!

Validation functions check each field against known constraints. Return None for valid, error string for invalid.

Validation alone is not enough. The LLM output might not even be valid JSON. We need a parsing layer that catches malformed output before validation runs.

07-error-handling.ipynb
python
# Parse and validate in one flow
def parse_command(llm_output: str):
    try:
        data = json.loads(llm_output)
    except json.JSONDecodeError:
        return None, "Invalid JSON. Please return valid JSON."

    error = validate_command(data)
    if error:
        return None, error

    return data, None  # Valid command

Parse JSON first (it might be malformed), then validate fields. Two layers of defense.

Validation pipeline

Fill in the blanks: Add validation to a tool handler

Loading practice…

Validate at system boundaries, where untrusted data enters your system. LLM output is a boundary. User input is a boundary. Internal function calls between your own code usually do not need validation. Focus on the edges.