Observability hooks for a running cluster

Observability is not a dashboard. It is three layers: logs you can search, metrics you can aggregate, and traces you can follow. For a RAG service, the retrieval path is where you will spend your debugging time, so instrument it first.

service.py (structured logging)
python
import structlog

log = structlog.get_logger()

class RAGService:
    def ingest(self, docs):
        log.info('ingest.start', count=len(docs), backend=self.embedder.backend)
        embeddings = self.embedder.embed_batch([d.text for d in docs])
        n = self.index.add(ids=[d.id for d in docs], texts=[d.text for d in docs], embeddings=embeddings)
        log.info('ingest.done', ingested=n)
        return {'ingested': n, 'embedder': self.embedder.backend}

Structured JSON logs with event names and fields beat printf logs every time. A log aggregator like Loki or Elasticsearch can filter on event=ingest.done or backend=ray without regex gymnastics.

sketch: prometheus metrics
python
from prometheus_client import Counter, Histogram, make_asgi_app

ingest_count = Counter('rag_ingest_total', 'Documents ingested', ['backend'])
query_latency = Histogram('rag_query_seconds', 'Query latency')

# In main.py:
app.mount('/metrics', make_asgi_app())

# In service.py:
ingest_count.labels(backend=self.embedder.backend).inc(n)
with query_latency.time():
    result = await answer_question(...)

A Prometheus ASGI mount plus a Counter and a Histogram give you the two metrics that matter: ingest throughput per backend and query latency distribution. Wire a ServiceMonitor if you run the Prometheus operator.

Quiz: Quiz

Loading practice…