Ship it: final checkpoint

You have the full pipeline running end to end. Before we celebrate, let us validate the finished app with a short retrieval smoke test and a final checkpoint that touches every stage.

test_retrieval.py
python
import pytest
from transcript import chunk_transcript
from keyword_index import build_keyword_index
from retrieval_fusion import fuse_and_get_top_k
from embedder import build_index, search_index


def _fake_transcript():
    return [
        {"text": "Welcome to the deep learning crash course.", "start": 0.0, "duration": 3.0},
        {"text": "We discuss gradient descent and its variants.", "start": 3.0, "duration": 3.0},
        {"text": "Karpathy explains the learning rate trick.", "start": 6.0, "duration": 3.0},
        {"text": "Finally we look at regularization strategies.", "start": 9.0, "duration": 3.0},
    ]


def test_hybrid_retrieves_named_entity():
    chunks = chunk_transcript(_fake_transcript(), chunk_size=5, overlap=1)
    index, _ = build_index(chunks)
    kw = build_keyword_index(chunks)

    keyword_hits = kw.search("Karpathy", top_k=3)
    vector_hits = search_index("Karpathy", index, chunks, top_k=3)
    fused = fuse_and_get_top_k(keyword_hits, vector_hits, top_k=2)

    assert any("Karpathy" in c["text"] for c in fused)

A minimal test that exercises every retrieval stage. Proper nouns like "Karpathy" are exactly where BM25 shines, so the fused top results should surface the right chunk.

terminal
bash
# Run the offline retrieval tests
make test

# Or run the app and try it yourself
make run

Two entry points. make test runs the offline retrieval tests for confidence. make run launches the Streamlit app so you can click through end to end.

Validation checklist: End to end checklist

Loading practice…

Checkpoint: Final pipeline checkpoint

Loading practice…