Policy PDF chunking

Welcome! I'm Param, and in this workshop we are going to build a supervisor-routed multi-agent system in LangGraph. One cheap router will classify every user message and dispatch to a specialist subagent that owns its domain. The reference app is an insurance copilot because it splits cleanly into policy, billing, claims, and escalation, but the pattern works for any domain where one assistant needs to answer from documents, from a database, and know when to hand off.

We start at the bottom of the stack. Before any agent can answer a policy question, we need a grounded knowledge base. That means taking real policy PDFs, extracting their text with layout-aware parsing, and chunking them into pieces small enough for retrieval but large enough to preserve meaning. Get this wrong and every subagent above it returns vague, uncited answers.

From PDF to searchable chunks

The path a policy document takes from disk to a queryable vector store.

rag_utils.py
python
from __future__ import annotations
import logging

logger = logging.getLogger(__name__)

CHUNK_SIZE = 800
CHUNK_OVERLAP = 120


def _load_pdf_markdown(path: str) -> str:
    """Extract PDF content as markdown so tables and headings survive."""
    try:
        import pymupdf4llm
        return pymupdf4llm.to_markdown(path)
    except Exception as exc:
        logger.warning('pymupdf4llm failed (%s). Falling back to plain read.', exc)
        with open(path, 'rb') as fh:
            return fh.read().decode('utf-8', errors='ignore')


def _split_text(text: str) -> list[str]:
    from langchain.text_splitter import RecursiveCharacterTextSplitter
    splitter = RecursiveCharacterTextSplitter(
        chunk_size=CHUNK_SIZE,
        chunk_overlap=CHUNK_OVERLAP,
        separators=['\n\n', '\n', '. ', ' ', ''],
    )
    return [c.strip() for c in splitter.split_text(text) if c.strip()]

pymupdf4llm extracts the PDF as markdown, which preserves headings and tables better than plain text extraction. The recursive splitter walks separator hierarchy so chunks break on paragraph boundaries first and only fall back to character-level splits when nothing else fits.

It is a tradeoff, not a universal answer. Policy language has long sentences and numbered clauses. At 800 characters you usually get one or two complete clauses per chunk, which is enough context for the LLM to answer a question without pulling in unrelated sections. The 120 character overlap protects against a single sentence getting split across two chunks and losing the connective tissue. For shorter, denser docs like FAQs, 400 with 60 overlap works better. Measure retrieval quality on your own corpus before you commit.

Quiz: Quiz

Loading practice…