Idempotency keys

Networks drop requests. Clients retry. Without idempotency keys, every retry creates a new job, burns LLM tokens, and confuses the user who now has three pending jobs for the same action. The fix is a header the client sends and the server remembers.

router.py

python

from fastapi import Header
from typing import Optional

@router.post("/jobs", response_model=JobStatus)
async def create_job(
    payload: JobStart,
    idempotency_key: Optional[str] = Header(default=None, alias="Idempotency-Key"),
):
    if idempotency_key:
        existing = await job_store.find_by_key(idempotency_key)
        if existing:
            return existing

    job_id = await job_store.create(idempotency_key=idempotency_key)
    task = asyncio.create_task(run_background_job(job_id, payload))
    await job_store.register_task(job_id, task)
    status = await job_store.get(job_id)
    return status

The client sends Idempotency-Key: <uuid> on every retry of the same logical job. On the server, we look it up first. If a job already exists for that key, we return it. Otherwise we create a new one and bind the key to the new job id.

Flashcards: Flashcards

Loading practice…

Quiz: Quiz

Loading practice…