Naive retrieval and first answer

The simplest possible retriever: embed the user question with the same model used for the documents, run a similarity search for top_k chunks, then ask the LLM to answer using only those chunks. We will call this the baseline. It works on easy questions and fails in predictable ways on harder ones.

rag_graph.py
python
async def _sub_embed(state: SubState) -> SubState:
    embeddings = await _embedder.generate_embeddings([state["query"]])
    return {"embedding": embeddings[0] if embeddings else []}


async def _sub_search(state: SubState) -> SubState:
    if not state.get("embedding"):
        return {"chunks": []}
    chunks = _vector_store.query(
        query_embedding=state["embedding"],
        n_results=state.get("top_k", 5),
    )
    return {"chunks": chunks}

These nodes form the child retriever graph. Embed the query, then search. You will wire the parent graph around this during the sub-graph phase, but for the baseline we call it directly.

rag_graph.py
python
async def _synthesize_node(state: GraphState) -> GraphState:
    contexts = state.get("contexts", []) or []
    question = state.get("anonymized_question") or state["question"]

    if not contexts:
        answer = "I couldn't find relevant information to answer that question."
    else:
        context_block = "\n\n".join(f"Source {i+1}:\n{c}" for i, c in enumerate(contexts[:10]))
        prompt = (
            "You are a careful assistant. Answer using ONLY the context below.\n"
            "If the context does not contain the answer, say you don't know.\n\n"
            f"Context:\n{context_block}\n\nQuestion: {question}\n\nAnswer:"
        )
        provider = get_llm_provider()
        collected = ""
        async for chunk in provider.generate_stream(prompt, temperature=0.2, max_tokens=800):
            collected += chunk
        answer = collected.strip()
    return {"answer": answer}

The synthesizer is strict: answer from the retrieved chunks only, and admit when the context does not contain the answer. Temperature is low because we want grounded facts, not creative prose.

terminal
bash
curl -X POST http://localhost:8000/advanced-rag/ask \
  -H "Content-Type: application/json" \
  -d '{
    "question": "What embedding model does the pipeline use?",
    "rewrite_n": 0,
    "anonymize": false,
    "with_evaluation": false
  }'

Setting rewrite_n to zero forces single-query retrieval. This is the baseline behavior, without any of the improvements we add later.

Quiz: Quiz

Loading practice…