The memory problem

Remember the SimpleAgent from earlier? It sends the entire conversation history with every call. After 50 turns, that could be thousands of tokens. After 200 turns, you might exceed the context window entirely. This is the memory problem.

Token count grows with every turn

05-memory-basics.ipynb

python

# Token usage grows linearly with conversation length
# Each turn adds ~50-200 tokens (user + assistant)

# Turn 1:  ~100 tokens sent
# Turn 10: ~1,000 tokens sent
# Turn 50: ~5,000 tokens sent
# Turn 100: ~10,000 tokens sent

# At $0.01/1K input tokens, a 100-turn conversation
# costs ~$0.10 PER CALL (and you make one per turn!)

Token costs grow quadratically: each turn sends more tokens, and you make a call every turn.

Quiz: Quiz

Loading practice…

Bigger context windows help, but they do not solve the problem. Larger windows cost more per call, are slower, and still have limits. A 200K token window sounds huge, but a busy agent with tools can fill it faster than you think. Smart memory management is always better than brute-forcing with bigger windows.