Sentence Transformers baseline

Before Ray, we need a baseline. The sequential embedder is a single SentenceTransformer instance encoding texts in-process. It is slow on large batches and perfectly fine for small ones. More importantly, it is the fallback the Ray-aware embedder falls back to when Ray is not installed.

pipeline/embed.py
python
from typing import List
import os

EMBEDDING_MODEL = os.getenv('EMBEDDING_MODEL', 'all-MiniLM-L6-v2')


class _SequentialEmbedder:
    def __init__(self, model_name: str = EMBEDDING_MODEL):
        from sentence_transformers import SentenceTransformer
        self.model_name = model_name
        self.model = SentenceTransformer(model_name)

    def encode(self, texts: List[str]) -> List[List[float]]:
        if not texts:
            return []
        vecs = self.model.encode(texts, convert_to_tensor=False, show_progress_bar=False)
        if hasattr(vecs, 'tolist'):
            vecs = vecs.tolist()
        return [list(v) for v in vecs]

all-MiniLM-L6-v2 is a small, fast model that produces 384-dimensional vectors. It is ideal for a workshop because the first run downloads in seconds and inference fits on CPU.

Local dev is the first reason. You want to run the tests and the API on your laptop without pinning Ray into every contributor environment. Graceful degradation is the other. If Ray fails to initialize inside a pod, the service should still answer queries on a single process rather than crash at startup.

Quiz: Quiz

Loading practice…