JSON repair pattern

Even with a strict prompt, LLMs occasionally return JSON wrapped in markdown fences, prefaced with "Sure, here is your result:", or truncated mid-object. Your parser needs to handle these gracefully. A single json.loads call on raw model output is a production incident waiting to happen.

agent/tasks.py

python

import json
import re
from typing import Any


def _extract_json(raw: str) -> Any:
    """Pull the first JSON object or array out of the model's response."""
    if not raw:
        raise ValueError("Empty response from LLM")

    # Strip markdown fences if present
    fenced = re.search(r"```(?:json)?\s*(\{.*?\}|\[.*?\])\s*```", raw, re.DOTALL)
    if fenced:
        return json.loads(fenced.group(1))

    # Try plain parse first
    try:
        return json.loads(raw)
    except json.JSONDecodeError:
        pass

    # Greedy scan for first balanced JSON value
    match = re.search(r"(\{.*\}|\[.*\])", raw, re.DOTALL)
    if not match:
        raise ValueError(f"Could not find JSON in response: {raw[:200]}")
    return json.loads(match.group(1))

Three layers. Markdown fence first, plain parse second, greedy object scan third. Most production LLM pipelines end up with a parser like this within a month of going live.

Ordering exercise: Arrange the repair attempts in the right order

Loading practice…

Code playground: Try the repair pattern

Loading practice…

Checkpoint: Structured output checkpoint

Loading practice…