Follow-up question handling across turns

Let us trace a real two-turn conversation to see the chain working. The opening question is direct. The follow-up references "it", which only makes sense with the previous turn in scope.

Two-turn conversation flow

See how history shapes the standalone query.

booktutor.py
python
chat_history = []

q1 = 'What is backpropagation?'
r1 = qa_system.invoke({'question': q1, 'chat_history': chat_history, 'book_name': 'Deep Learning'})
chat_history.append((q1, r1['answer']))

q2 = 'How does it handle vanishing gradients?'
r2 = qa_system.invoke({'question': q2, 'chat_history': chat_history, 'book_name': 'Deep Learning'})
chat_history.append((q2, r2['answer']))

# Under the hood, the chain rewrites q2 into something like:
# How does backpropagation handle vanishing gradients?
# That rewritten query is what hits FAISS.

Each append adds context the chain uses to rewrite the next question into a standalone query. You never have to implement the rewrite yourself; the chain handles it.

Two problems show up. First, the rewrite call gets expensive because it reads the whole history every turn. Second, the rewriter can lose focus with very long histories and produce worse standalone queries. The fix is to summarize older turns or keep only the recent ones. LangChain has ConversationSummaryBufferMemory for exactly this, but a simple sliding window is often enough.

Quiz: Quiz

Loading practice…