Measured speedup
Parallel embedding is not free. Ray adds serialization overhead on top of each remote call. The win only shows up when the per-batch work is large enough to dwarf that overhead. Measure before you celebrate.
import time
from pipeline.embed import RayEmbedder, _SequentialEmbedder
TEXTS = [
f'Document number {i}: lorem ipsum dolor sit amet, consectetur adipiscing elit.'
for i in range(1000)
]
seq = _SequentialEmbedder()
t0 = time.perf_counter()
seq.encode(TEXTS)
seq_time = time.perf_counter() - t0
ray_embedder = RayEmbedder(num_workers=4)
t0 = time.perf_counter()
ray_embedder.embed_batch(TEXTS)
ray_time = time.perf_counter() - t0
print(f'sequential: {seq_time:.2f}s')
print(f'ray workers=4: {ray_time:.2f}s')
print(f'speedup: {seq_time / ray_time:.2f}x')A quick harness. Use a real batch size that matches your ingest traffic. On a four-core laptop with four workers, expect a speedup in the two to three range on 1000 documents. More workers than cores does not help.
Physical cores are the ceiling. Going past core count just time-slices. Ray also serializes inputs and outputs across processes, so small batches pay overhead you cannot reclaim. And SentenceTransformer itself vectorizes well inside one process, so you are comparing against a fairly fast baseline.
Quiz: Quiz
Loading practice…
Checkpoint: Ray embedding checkpoint
Loading practice…