Environment setup
Welcome! I'm Param, and in this course we are going to build an invoice parser that extracts structured data from images using vision LLMs. By the end, you will have a FastAPI service that accepts an invoice upload, sends it to a multimodal model, and returns typed, validated data with confidence scores.
The project takes a real-world problem: you receive invoices as images or PDFs, and you need to extract vendor name, line items, totals, currency, and tax amounts into structured data. Instead of writing brittle parsers, we will let a vision LLM read the document and return exactly the fields we define in a Pydantic schema.
# Clone the workshop repository
git clone https://github.com/learnwithparam/vision-llm-extraction.git
cd vision-llm-extraction
# One command to set up everything
make devThis clones the repo, creates a virtual environment with uv, installs dependencies, and starts the FastAPI server.
# Copy the example and add your API key
cp .env.example .env
# Configure your vision LLM provider:
VISION_LLM_PROVIDER=gemini
GEMINI_API_KEY=your_key_here
VISION_MODEL=gemini-2.0-flashThe project supports Gemini, OpenRouter, and Fireworks for vision tasks. Gemini is recommended because it handles PDF input natively.
Validation checklist: Environment setup checklist
Loading practice…