Parallelization

Parallelization lets you run multiple LLM calls at the same time instead of waiting for each one to finish. This is ideal when sub-tasks are independent, like analyzing different sections of a document or getting multiple perspectives on a question.

Sequential processing (slow)

Parallel execution completes all tasks in the time of the slowest one.

Parallel processing (fast)

Only when sub-tasks are independent. If Task B needs the output of Task A, you cannot run them in parallel. The sweet spot is tasks like multi-perspective analysis or batch processing where each call is self-contained.

asyncio.gather() runs multiple async functions simultaneously and waits for all of them to finish. This is how we parallelize LLM calls. Instead of waiting for each one sequentially (3 calls x 3 seconds = 9 seconds), we run them all at once (3 seconds total).

patterns/03_parallelization.py
python
import asyncio

async def analyze_aspect(aspect, topic, llm):
    """Analyze one aspect of a topic."""
    prompt = f"Analyze the {aspect} of {topic}. Be specific."
    return llm.generate(prompt).content

async def parallel_analysis(topic, llm):
    """Run multiple analyses in parallel."""
    aspects = ["benefits", "challenges", "future trends"]

    # Create tasks for parallel execution
    tasks = [
        analyze_aspect(aspect, topic, llm)
        for aspect in aspects
    ]

    # Run all tasks concurrently
    results = await asyncio.gather(*tasks)

    # Aggregate results
    combined = "\n".join(
        f"{aspect}: {result}"
        for aspect, result in zip(aspects, results)
    )
    return combined

# Run the parallel analysis
result = asyncio.run(parallel_analysis("AI agents", llm))

Use asyncio to run multiple LLM calls concurrently and gather results.

It genuinely runs faster when tasks are independent. If you have 3 API calls that each take 2 seconds, running them sequentially takes 6 seconds. Running them in parallel takes about 2 seconds. The key requirement is that the tasks cannot depend on each other's results.

Quiz: Quiz

Loading practice…

Matching exercise: Match parallelization concepts

Loading practice…

Flashcards: Flashcards

Loading practice…

Parallelization is your first performance optimization pattern. When tasks are independent, running them concurrently can dramatically reduce wait times. Next, we will learn reflection, where agents critique and improve their own output.