Generators, scope and yield

When you use ChatGPT and text appears word by word, that is a generator in action. Generators produce values one at a time using yield instead of building a whole list in memory. They are the pattern behind streaming LLM responses.

Generator vs list memory

How generators use constant memory compared to lists

generators_and_scope.py
python
# Variable scope: local vs global
model = "gpt-4o"  # global variable

def switch_model():
    model = "claude-3.5-sonnet"  # LOCAL variable, does not change global
    print(f"Inside function: {model}")

switch_model()
print(f"Outside function: {model}")  # still "gpt-4o"

# Better pattern: use a mutable container
state = {"count": 0}

def track_request():
    state["count"] += 1  # modifying dict contents does not need global

track_request()
track_request()
print(f"Requests tracked: {state['count']}")  # 2

# Closures: a function that remembers its environment
def make_logger(prefix):
    def log(message):
        print(f"[{prefix}] {message}")
    return log

api_log = make_logger("API")
api_log("Request sent")  # [API] Request sent

Variables inside functions are local by default. Closures let inner functions remember values from their enclosing scope. This is the same pattern used by decorators.

return sends a value and exits the function forever. yield sends a value and pauses the function. When you ask for the next value, it resumes right where it left off. This means generators can produce an infinite stream of values without using memory for all of them at once.

generators_and_scope.py
python
# Generators with yield
def count_up_to(n):
    i = 1
    while i <= n:
        yield i
        i += 1

for num in count_up_to(5):
    print(num, end=" ")  # 1 2 3 4 5
print()

# Memory efficiency
big_list = [x * x for x in range(1000)]   # 1000 items in memory
big_gen = (x * x for x in range(1000))    # almost no memory
print(f"First: {next(big_gen)}")  # 0
print(f"Second: {next(big_gen)}") # 1

# Simulating streaming LLM responses
def stream_response(text):
    words = text.split()
    for word in words:
        yield word + " "

for token in stream_response("Python generators are perfect for streaming"):
    print(token, end="", flush=True)
print()

Generators use yield to produce values lazily. Generator expressions use () instead of []. next() gets one value at a time. This is exactly how LLM streaming works.

Quiz: Quiz

Loading practice…

Fill in the blanks: Complete the generator

Loading practice…

Validation checklist: Generators checklist

Loading practice…

Checkpoint: Advanced patterns checkpoint

Loading practice…

Timed quiz: Advanced patterns speed round

Loading practice…