Tuning retrieval quality

Retrieval quality is what separates a RAG demo from a RAG product. The pipeline is only as good as the chunks it surfaces. Two small changes, bigger candidate pool and reranking, often turn mediocre answers into sharp ones.

It depends on how noisy your index is. 20 candidates is a solid default. If your content is highly uniform (a single product manual), 10 is often enough. If your index covers many topics and styles, bump it to 30 or 50. The cost is pure CPU on the cross-encoder, which is cheap. The win is better recall before reranking locks in precision.