Re-rank hook

Vector search is a strong baseline, but on larger corpora you will want a second stage: pull a wider candidate set, then re-rank it with a stronger model. Building that in from day one is overkill. The trick is to leave a hook, so adding a re-ranker later is a one-file change instead of a refactor.

layers/retrieval.py (rerank hook)
python
from typing import Callable, List, Optional


ReRanker = Callable[[str, List[RetrievedDoc]], List[RetrievedDoc]]


class RetrievalLayer:
    def __init__(
        self,
        collection_name: str = "prod_agent_kb",
        reranker: Optional[ReRanker] = None,
    ) -> None:
        self._reranker = reranker
        # ... existing init ...

    def search_with_ids(
        self, query: str, k: int = 3, candidate_k: int = 10
    ) -> List[RetrievedDoc]:
        if not self.available or self._collection is None:
            return []

        # Pull a wider candidate pool when a reranker is present.
        pool_size = candidate_k if self._reranker else k
        candidates = self._query(query, pool_size)

        if self._reranker is not None:
            candidates = self._reranker(query, candidates)

        return candidates[:k]

When no reranker is configured, the layer queries for exactly k documents and returns them. When a reranker is present, it pulls a wider pool and lets the reranker pick the best k. The caller interface never changes.

Retrieve-then-rerank pipeline

Fast recall with embeddings, precise selection with a cross-encoder or LLM re-ranker.

When evaluation data shows that your top-k results contain the right answer but in the wrong order. That is the signature of a re-rank gap. If the right answer is nowhere in the candidate pool, a re-ranker will not save you; you need better embeddings, better chunking, or better queries. Re-ranking is a precision improvement, not a recall one.

Checkpoint: Retrieval layer checkpoint

Loading practice…