BackgroundTasks vs asyncio queue

Not every LLM call should be a streaming request. Some tasks take minutes, like generating a report or running an eval. The user kicks off the work and polls for the result later. FastAPI gives you two primitives here, and picking the wrong one is a common production bug.

BackgroundTasks vs asyncio task

Two different lifetimes, two different places to run work.

router.py
python
import asyncio
from fastapi import APIRouter, HTTPException
from models import JobStart, JobStatus
from job_store import job_store
from service import run_background_job

@router.post("/jobs", response_model=JobStatus)
async def create_job(payload: JobStart):
    job_id = await job_store.create()
    task = asyncio.create_task(run_background_job(job_id, payload))
    await job_store.register_task(job_id, task)
    status = await job_store.get(job_id)
    if not status:
        raise HTTPException(status_code=500, detail="job_store lost the job")
    return status

@router.get("/jobs/{job_id}", response_model=JobStatus)
async def get_job(job_id: str):
    status = await job_store.get(job_id)
    if not status:
        raise HTTPException(status_code=404, detail="job not found")
    return status

asyncio.create_task kicks off the worker coroutine concurrently with the request, returns a task handle we can track, and lets the endpoint respond immediately with a pending status. FastAPI BackgroundTasks runs after the response is sent, which is fine for short cleanup, not for long LLM work you want to poll.

Because a five minute LLM job becomes a five minute HTTP request, and every proxy in between will kill it. The client also has to hold the connection open the entire time. Decoupling into create-then-poll lets the request return in milliseconds and the client check progress on its own schedule.

Quiz: Quiz

Loading practice…