Local embeddings with sentence-transformers
The original demo called OpenAI for text-embedding-ada-002. That is two API keys a student has to create, pay for, and wire up correctly. We swap that for sentence-transformers' all-MiniLM-L6-v2, which runs locally on CPU and ships inside ChromaDB's utility package.
from chromadb.utils import embedding_functions
self.embedding_function = embedding_functions.SentenceTransformerEmbeddingFunction(
model_name=EMBEDDING_MODEL, # "all-MiniLM-L6-v2" by default
)
self.collections[role] = self.chroma_client.create_collection(
name=f"{role}_docs",
embedding_function=self.embedding_function,
)
ChromaDB ships a SentenceTransformerEmbeddingFunction that downloads the model on first use and caches it. Zero API keys involved in the embedding step.
The trade-off is small and known. all-MiniLM-L6-v2 is smaller than ada-002, so retrieval quality on very long or very technical documents drops a little. For a workshop where every doc fits in a screen, the difference is invisible. The onboarding win, one API key instead of two, shows up immediately.
AI prompt: Swap the embedding model
Loading practice…
Quiz: Quiz
Loading practice…