Visualizing RAG pipelines with LangGraph StateGraph

Your RAG pipeline is a mental model nobody shares

Your RAG pipeline has 7 steps: query rewrite, vector retrieve, rerank, quote extract, grade, generate, and fallback. You know how it fits together because you built it. Your teammate, who is debugging a bad answer at 2am, does not. They read your 400-line Python file, trace the function calls, and eventually ask you over Slack what the flow is supposed to be.

This is the cost of pipeline logic living only as code. Graphs are the natural shape of multi-step pipelines, and most teams draw the graph on a whiteboard once, then let the code drift away from the diagram. LangGraph flips this around: the graph IS the code. When you change a node, the diagram updates. When you onboard someone new, they look at the graph and the code lines up.

This post is how to model a RAG pipeline as a LangGraph StateGraph, the state schema that carries information between nodes, the rendering trick that gives you a diagram, and the debugging workflow that makes bad answers easy to trace.

Why does modeling RAG as a graph help?

Because RAG pipelines have branches, loops, and conditional steps that do not fit a linear function call chain. A straight-line script hides the routing decisions inside if branches. A graph makes the routing explicit and inspectable.

3 concrete benefits:

1. You can see the whole flow at once. A 10-node graph is a 10-node image. A 400-line script is a 400-line file. The image is faster to understand and survives refactoring better.
2. Debugging becomes structural. When an answer is wrong, you look at which node the state flowed through and which node modified which field. This is impossible to see in a nested Python script.
3. Iteration is cheaper. Adding a reranker means adding one node and one edge. Adding a reranker to a nested script means touching half a dozen places.

graph TD
    Q[Question] --> Rewrite[rewrite]
    Rewrite --> Retrieve[retrieve]
    Retrieve --> Rerank[rerank]
    Rerank --> Extract[extract quotes]
    Extract --> Grade{grade}
    Grade -->|good| Generate[generate]
    Grade -->|bad| Rewrite
    Generate --> End([answer])

    style Grade fill:#fef3c7,stroke:#b45309
    style Generate fill:#dcfce7,stroke:#15803d

That is a real production RAG pipeline. Every node is a Python function. Every edge is an edge in a LangGraph definition. The diagram and the code are the same thing.

What is a StateGraph in LangGraph?

A StateGraph is a graph where nodes are functions that read and update a shared state dict, and edges are the transitions between nodes. LangGraph provides the graph construction, the conditional edge routing, and the execution engine. You provide the state schema and the node functions.

The state is the carrier. Every node reads from it and writes partial updates back. LangGraph merges the updates into the next state automatically. This is the same pattern as the agentic RAG graph in the Agentic RAG with LangGraph: Planning, Rewriting, and Tool Use post, applied to any pipeline shape.

# filename: state.py
# description: The state that flows through a RAG pipeline. Every node
# reads and writes fields on this type.
from typing import TypedDict

class RagState(TypedDict):
    question: str
    rewritten: str
    retrieved: list[str]
    reranked: list[str]
    quotes: list[str]
    grade: str
    answer: str

Keep the state flat. Nested state is harder to inspect when debugging. Every field is a column in your mental spreadsheet of "what the pipeline knew at step N."

How do you define the nodes and wire them up?

Each node is a function that takes state and returns a partial dict. The graph builder wires them with add_node and add_edge calls.

# filename: rag_graph.py
# description: Build a RAG pipeline as a LangGraph StateGraph.
# Every step is a node; every transition is an edge.
from langgraph.graph import StateGraph, END
from state import RagState
from nodes import rewrite, retrieve, rerank, extract, grade, generate

builder = StateGraph(RagState)
builder.add_node('rewrite', rewrite)
builder.add_node('retrieve', retrieve)
builder.add_node('rerank', rerank)
builder.add_node('extract', extract)
builder.add_node('grade', grade)
builder.add_node('generate', generate)

builder.set_entry_point('rewrite')
builder.add_edge('rewrite', 'retrieve')
builder.add_edge('retrieve', 'rerank')
builder.add_edge('rerank', 'extract')
builder.add_edge('extract', 'grade')


def route_after_grade(state: RagState) -> str:
    return 'generate' if state['grade'] == 'good' else 'rewrite'


builder.add_conditional_edges('grade', route_after_grade, {
    'generate': 'generate',
    'rewrite': 'rewrite',
})
builder.add_edge('generate', END)

graph = builder.compile()

Read the add_conditional_edges call. That is where the loop back to rewrite lives when the grade is bad. A linear script would hide this in an if/else deep inside a function. The graph shows it at the wiring layer where you can see it immediately.

How do you render the graph as an image?

LangGraph has graph.get_graph().draw_mermaid() or draw_png() which produce a diagram from the compiled graph. You can drop the result into your docs, your wiki, or your PR description.

# filename: render.py
# description: Render the compiled graph to Mermaid syntax or a PNG.
from rag_graph import graph

# Mermaid for docs
print(graph.get_graph().draw_mermaid())

# PNG for a file
with open('rag_graph.png', 'wb') as fh:
    fh.write(graph.get_graph().draw_png())

The Mermaid output drops directly into a GitHub README or blog post. The PNG is good for slide decks or Notion pages. Either way, the diagram is generated from the code, not maintained separately, so it never drifts.

This is the killer feature for team communication. A diagram that regenerates on every commit is trustworthy in a way a whiteboard sketch is not.

How do you debug a bad answer with the graph?

Run the graph with streaming state updates and inspect what each node wrote. LangGraph exposes the per-node state transitions, which gives you a timeline of what the pipeline knew at each step.

# filename: debug_run.py
# description: Stream state transitions to see what each node did.
from rag_graph import graph

initial = {
    'question': 'which services were affected by the outage?',
    'rewritten': '',
    'retrieved': [],
    'reranked': [],
    'quotes': [],
    'grade': '',
    'answer': '',
}

for step in graph.stream(initial):
    for node_name, state_update in step.items():
        print(f'--- {node_name} ---')
        for key, value in state_update.items():
            short = str(value)[:120]
            print(f'  {key}: {short}')

The stream output shows you exactly what every node wrote. If the retrieval step returned empty lists, the pipeline is broken at retrieval. If retrieval returned chunks but reranking dropped them all, the reranker is too aggressive. If everything looks fine until generation, the generator prompt is the bug. You can point at the exact node without reading the whole codebase.

This is the moment you realize that the graph abstraction is not just pretty. It is the single most effective debugging tool in a multi-step RAG pipeline.

For the full agentic RAG pattern with planning and tool use, see the Agentic RAG with LangGraph: Planning, Rewriting, and Tool Use post. For the evaluation loop that catches the bad answers in the first place, see RAGAS Evaluation for RAG Pipelines: A Practical Guide.

How do you add checkpointing for resumable pipelines?

LangGraph's checkpointer stores state snapshots between nodes so you can resume a pipeline from any point. Useful for long-running pipelines, human-in-the-loop approvals, and debugging reruns with the exact state from a failed production request.

The checkpointer is a 1-line addition to the compile step. You pass a SqliteSaver or AsyncPostgresSaver and every node transition is persisted:

# filename: checkpointed.py
# description: Compile the graph with a persistence layer so state
# can be resumed from any step.
from langgraph.checkpoint.postgres import PostgresSaver

checkpointer = PostgresSaver.from_conn_string('postgresql://...')
graph = builder.compile(checkpointer=checkpointer)

# Use a thread_id to scope the state for resumption
config = {'configurable': {'thread_id': 'request-123'}}
result = graph.invoke(initial_state, config)

For the deeper dive on the persistence layer specifically, see the LangGraph Persistence: Why Production Agents Need Thread Models post.

What to do Monday morning

1. Sketch your current RAG pipeline on paper. Count the steps and the branches. If you have more than 5 steps or any branch, you benefit from converting to a StateGraph.
2. Create a RagState TypedDict with every field your pipeline needs to pass between steps. Keep it flat, one field per concept.
3. Wrap each current step as a node function that takes state and returns a partial dict. Most existing RAG code can be converted with minimal changes.
4. Build the graph with StateGraph, add edges, compile. Run draw_mermaid() and check the diagram against your sketch. Any mismatch is a bug in your mental model.
5. Stream a real query through the graph with graph.stream() and watch the per-node state updates. This is the new debugging workflow; the old print-statement chain is obsolete.

The headline: LangGraph turns a multi-step RAG pipeline into a diagrammable, debuggable graph. The code and the diagram are the same thing, which eliminates drift. Ship it before your next debugging session at 2am.

Frequently asked questions

What is a StateGraph in LangGraph?

A StateGraph is a graph abstraction where nodes are functions that read and update a shared state, and edges define the transitions between nodes. LangGraph provides the execution engine, conditional routing, and diagram rendering. You provide the state schema and the node functions. It is the natural shape for multi-step pipelines like RAG and agent loops.

Why should I model a RAG pipeline as a graph instead of a script?

Because multi-step pipelines have branches, loops, and conditional transitions that do not fit linear function calls. A graph makes the routing explicit, gives you a diagram you can regenerate from code, and provides per-node state inspection for debugging. The 400-line script becomes a 10-node image plus 10 small node functions.

How do I render a LangGraph pipeline as a diagram?

Use graph.get_graph().draw_mermaid() for Mermaid syntax you can drop into docs, or draw_png() for a rendered image. Both are generated from the compiled graph, so the diagram never drifts from the code. This is the killer feature for team communication and onboarding.

How does LangGraph help debug a bad RAG answer?

By letting you stream per-node state updates with graph.stream(). You can see exactly what each node wrote to state, which makes it obvious where a bad answer came from. Retrieval returning empty lists, a reranker dropping everything, or a generator producing wrong output all become immediately visible in the stream.

What is checkpointing in LangGraph and why does it matter for RAG?

Checkpointing persists the state between nodes so you can resume a pipeline from any point. For RAG, this enables long-running pipelines, human-in-the-loop approvals, and the ability to replay a failed production request with its exact state. Add a PostgresSaver or SqliteSaver at compile time and every transition is persisted automatically.

Key takeaways

1. RAG pipelines have branches and loops that do not fit linear scripts. A graph abstraction makes the shape explicit and debuggable.
2. LangGraph StateGraph lets you define state, nodes, and edges in code. The graph and the code are the same thing, so diagrams never drift.
3. Keep state flat. One field per concept. Nested state hides information during debugging and complicates transitions.
4. Render the graph with draw_mermaid() or draw_png() for docs and onboarding. Regenerating the diagram from code eliminates stale whiteboard sketches.
5. Debug with graph.stream() to inspect per-node state updates. This is the workflow that replaces scattered print statements.
6. To see StateGraph wired into a full production agentic RAG stack with planning, reranking, and self-correction, walk through the Agentic RAG Masterclass, or start with the RAG Fundamentals primer.

For the full LangGraph documentation on StateGraph construction, checkpointers, and visualization, see the LangGraph docs. The StateGraph API reference there documents every method used in this post.