Memory buffer and standalone question rewriting

ConversationalRetrievalChain runs two LLM calls per turn. First, it combines chat history plus the latest question into a standalone query. Then it retrieves against that query, stuffs the chunks into your grounding prompt, and asks the LLM for the final answer.

booktutor.py

python

from langchain.chains import ConversationalRetrievalChain
from langchain_openai import ChatOpenAI

retriever = vectorstore.as_retriever(
    search_type='mmr',
    search_kwargs={'k': 5},
)

llm = ChatOpenAI(
    model=os.environ.get('LLM_MODEL', 'google/gemini-2.5-flash-lite'),
    openai_api_base=os.environ.get('OPENAI_API_BASE'),
    openai_api_key=os.environ.get('OPENAI_API_KEY'),
    temperature=0,
    max_tokens=1024,
)

qa_chain = ConversationalRetrievalChain.from_llm(
    llm=llm,
    retriever=retriever,
    return_source_documents=True,
    combine_docs_chain_kwargs={
        'prompt': prompt,
        'document_variable_name': 'context',
    },
)

MMR retrieval (maximal marginal relevance) diversifies results, so you get chunks covering different aspects of the question. temperature=0 keeps answers deterministic. combine_docs_chain_kwargs wires in the grounding prompt from the previous lesson.

Ordering exercise: Put the chain steps in order

Loading practice…

MMR picks chunks that are both relevant and diverse. Plain similarity can return five near-duplicate chunks from the same paragraph, giving the LLM redundant context. MMR spreads the selection across different parts of the document, which helps when a question is answered across multiple sections.