Per-subquestion retrieval and fan-in

Fan-in is the part most people get wrong. You invoke retrieval per rewrite, get back overlapping chunk sets, and need to merge them without feeding three copies of the same chunk to the synthesizer. The parent node handles this with a simple seen-set keyed on source, chunk_index, and a content prefix.

rag_graph.py
python
def _build_sub_graph():
    """Compile the child retriever graph once."""
    g = StateGraph(SubState)
    g.add_node("embed", _sub_embed)
    g.add_node("search", _sub_search)
    g.set_entry_point("embed")
    g.add_edge("embed", "search")
    g.add_edge("search", END)
    return g.compile()

_sub_graph = _build_sub_graph()

A small child graph, embed then search, ending at END. Compiling once at import avoids paying the compile cost per request.

rag_graph.py
python
async def _sub_retrieve_node(state: GraphState) -> GraphState:
    top_k = state.get("top_k", 5)
    rewrites = state.get("rewrites") or [state.get("anonymized_question") or state["question"]]

    all_chunks: list[DocumentChunk] = []
    seen_ids = set()

    for rq in rewrites:
        try:
            result = await _sub_graph.ainvoke({"query": rq, "top_k": top_k})
        except Exception as e:
            logger.warning(f"Sub-graph failed for query '{rq}': {e}")
            continue
        for chunk in result.get("chunks", []) or []:
            key = (chunk.source, chunk.chunk_index, chunk.content[:80])
            if key in seen_ids:
                continue
            seen_ids.add(key)
            all_chunks.append(chunk)

    contexts = [c.content for c in all_chunks]
    return {"retrieved": all_chunks, "contexts": contexts}

Habits worth copying. Catch sub-graph exceptions so one bad rewrite does not kill the request. Key the seen-set on (source, chunk_index, prefix) so near-duplicates are caught. Never mutate state, always return a new dict.

Some ingestion paths produce chunks with identical source and chunk_index when a document was re-indexed or when chunking logic changed. The content prefix is a cheap tiebreaker that distinguishes true duplicates from accidental collisions without hashing the full chunk body. In production you might swap this for a real content hash.

Quiz: Quiz

Loading practice…

Checkpoint: Sub-graph checkpoint

Loading practice…