Load test the ingest path

An HPA you never stress is theatre. We push a thousand documents through ingest in batches, watch kubectl get hpa, and confirm the replica count moves. If it does not, the bottleneck is somewhere else and the metric picked the wrong thing to watch.

scripts/load_test.py

python

import asyncio
import httpx

BASE = 'http://localhost:8000/enterprise-rag'
TOTAL = 1000
BATCH = 50


async def push_batch(client, start: int) -> None:
    docs = [
        {'id': f'doc-{i}', 'text': f'Load test document number {i}. ' * 20}
        for i in range(start, start + BATCH)
    ]
    r = await client.post(f'{BASE}/ingest', json={'docs': docs}, timeout=120)
    r.raise_for_status()


async def main() -> None:
    async with httpx.AsyncClient() as client:
        tasks = [push_batch(client, i) for i in range(0, TOTAL, BATCH)]
        await asyncio.gather(*tasks)


if __name__ == '__main__':
    asyncio.run(main())

Parallel async batches drive CPU usage inside the embedding pods. Watch the HPA while this runs.

terminal

bash

# Port-forward the Service
kubectl port-forward svc/enterprise-rag 8000:80 &

# Watch the HPA while the load test runs
watch -n 2 kubectl get hpa,pods

# In another terminal
python scripts/load_test.py

You should see CPU percent climb, replicas scale out toward maxReplicas, ingest throughput rise, then scale back down after the stabilization window.

Quiz: Quiz

Loading practice…