Async, await and concurrency

AI API calls take time (network I/O, model processing), and async lets you do other work while waiting. Instead of calling APIs one-by-one (4 seconds total), you can call them in parallel (2 seconds total). Every modern AI app uses async.

Sync vs async timeline

Sequential calls vs parallel execution with async/await

async_await.py
python
import asyncio

# Basic async function
async def fetch_model_response(query):
    """Simulates an API call that takes time."""
    print(f"  Sending: {query}")
    await asyncio.sleep(0.5)  # Simulates network delay
    return f"Response for: {query}"

# Running an async function
async def main():
    result = await fetch_model_response("What are top products?")
    print(f"  Got: {result}")

asyncio.run(main())

async def creates a coroutine. await pauses until the result is ready. asyncio.run() starts the event loop. This is the foundation of non-blocking AI code.

Sync code waits for each operation to finish before starting the next. Async code can start multiple operations and process results as they arrive. Use async when you have I/O-bound work (API calls, file reads, database queries). Use sync for CPU-bound work (number crunching, data processing).

async_await.py
python
import asyncio

async def fetch_model_response(query):
    await asyncio.sleep(0.5)
    return f"Response for: {query}"

# Running multiple async calls in parallel
async def parallel_demo():
    queries = ["query 1", "query 2", "query 3"]

    # gather() runs all tasks at the same time
    results = await asyncio.gather(
        *[fetch_model_response(q) for q in queries]
    )

    for r in results:
        print(f"  {r}")

asyncio.run(parallel_demo())

# Async generators (async for)
async def stream_events():
    events = ["guardrails", "sql_agent", "result"]
    for event in events:
        await asyncio.sleep(0.1)
        yield event  # yields one at a time

async def process():
    async for event in stream_events():
        print(f"  Event: {event}")

asyncio.run(process())

asyncio.gather() runs multiple async calls concurrently, and async for iterates over an async generator, which is how you stream events from an AI agent.

AI prompt: Try it with AI

Loading practice…

Quiz: Quiz

Loading practice…

Ordering exercise: Async flow steps

Loading practice…

Validation checklist: Async checklist

Loading practice…

Hints: Hints

Loading practice…