A retrieval prompt that refuses to hallucinate
Retrieval gives the LLM a pile of relevant chunks. A grounding prompt tells the LLM how to use them. Without a careful prompt, the model mixes retrieved context with its prior knowledge and quietly slips hallucinations into the answer.
Where the grounding prompt sits
The prompt is the final layer before the answer LLM.
from langchain_core.prompts import PromptTemplate
template = '''You are a helpful assistant answering questions about the book: {book_name}.
Use the following context to answer the question: {context}
Question: {question}
Answer the question accurately and concisely based on the context provided.'''
prompt = PromptTemplate(
input_variables=['book_name', 'context', 'question'],
template=template,
)Three variables: book_name for identity, context for retrieved chunks, and question for the current (already rewritten) user turn. Explicitly telling the model to answer based on the context provided is the phrase that drops hallucinations.
Teach the model to say so. Add a line like "If the context does not contain the answer, say you do not know rather than guessing." That single instruction is the difference between a tutor you can trust and one that makes things up when it is unsure. You can also raise retrieval k to widen the net before giving up.
Quiz: Quiz
Loading practice…