The schema

Sending an image to a vision LLM and getting raw text back is useful, but unpredictable. The model might return a paragraph description one time and a bullet list the next. Pydantic schemas solve this by defining the exact shape of the data you expect. You instruct the model to return JSON matching your schema, then validate the response before it goes anywhere.

invoice_analyzer.py
python
from pydantic import BaseModel, Field
from typing import List, Optional

class InvoiceItem(BaseModel):
    """A single line item on an invoice"""
    description: str = Field(description="Description of the product or service")
    quantity: Optional[float] = Field(None, description="Quantity of the item")
    unit_price: Optional[float] = Field(None, description="Price per unit")
    amount: float = Field(description="Total amount for this line item")

class InvoiceData(BaseModel):
    """Structured data extracted from an invoice"""
    is_invoice: bool = Field(description="Whether the document is actually an invoice")
    vendor_name: Optional[str] = Field(None, description="Name of the vendor")
    invoice_number: Optional[str] = Field(None, description="Unique invoice identifier")
    invoice_date: Optional[str] = Field(None, description="Date issued (YYYY-MM-DD)")
    due_date: Optional[str] = Field(None, description="Payment due date (YYYY-MM-DD)")
    items: List[InvoiceItem] = Field(default_factory=list, description="Line items")
    subtotal: Optional[float] = Field(None, description="Total before taxes")
    tax_amount: Optional[float] = Field(None, description="Tax amount")
    total_amount: Optional[float] = Field(None, description="Final total amount due")
    currency: Optional[str] = Field("USD", description="Currency code (USD, EUR, GBP)")

Every field has a type, a default, and a description. The descriptions serve double duty: they document the schema for developers, and they guide the LLM when you include the schema in the extraction prompt.

invoice_analyzer.py
python
prompt = """
You are an expert invoice processing agent.

1. Determine if this document is an invoice or similar billing document.
2. If it IS an invoice, extract all relevant fields accurately.
3. If it IS NOT an invoice, set is_invoice to false and return empty values.

Extract these fields in JSON format:
- is_invoice: boolean
- vendor_name: string
- invoice_number: string
- invoice_date: string (YYYY-MM-DD format)
- due_date: string (YYYY-MM-DD format)
- items: list of {description, quantity, unit_price, amount}
- subtotal: number
- tax_amount: number
- total_amount: number
- currency: string (3-letter code, default USD)

Return ONLY the JSON object.
"""

The prompt lists every field the model should extract. The "Return ONLY the JSON object" instruction prevents the model from wrapping the JSON in explanatory text.

invoice_analyzer.py
python
# Clean up response text if it contains markdown code blocks
clean_json = response_text.strip()
if clean_json.startswith("```json"):
    clean_json = clean_json.replace("```json", "", 1)
if clean_json.endswith("```"):
    clean_json = clean_json.rsplit("```", 1)[0]
clean_json = clean_json.strip()

# Parse and validate with Pydantic
data = json.loads(clean_json)
return InvoiceData(**data)

Even with "Return ONLY the JSON" in the prompt, models sometimes wrap the response in markdown code fences. The cleanup step handles this gracefully. Then Pydantic validates the parsed data against your schema.

Flashcards: Flashcards

Loading practice…

Quiz: Quiz

Loading practice…

AI prompt: Try it: vision extraction

Loading practice…

Checkpoint: Vision fundamentals checkpoint

Loading practice…