Ray actors overview
Ray is a distributed compute framework with a simple mental model: actors are classes that live in other processes, methods on them return futures, and ray.get resolves those futures. For embedding, each actor holds its own SentenceTransformer instance and encodes a shard of the batch in parallel.
Matching exercise: Ray vocabulary
Loading practice…
def _try_import_ray():
try:
import ray
return ray
except Exception as exc:
logger.info('Ray not available (%s) - using sequential embedder', exc)
return NoneA lazy import keeps Ray optional. If the import fails, the service logs it and continues with the sequential embedder. You can run the same code on a laptop with no Ray installed and on a cluster with Ray running.
Embedding carries a heavy, stateful resource: the model weights. A ray task would reload the model on every call. An actor loads the model once inside its process and reuses it across calls. That difference turns parallel embedding from a nice idea into something that actually helps.
Quiz: Quiz
Loading practice…