From stateless to stateful

Here is the fundamental truth about LLMs: they have no memory. Every API call is completely independent. The model does not remember what you said 5 seconds ago.

That memory is built by the application layer, not the model itself. ChatGPT stores your conversation history in a database and sends it with each API call. The raw LLM underneath has zero memory. Let me prove it with two API calls.

01-agent-fundamentals.ipynb
python
from litellm import completion

# First call: introduce yourself
response = completion(
    model="gemini/gemini-2.0-flash-exp",
    messages=[{"role": "user", "content": "My name is Alex."}]
)
print(response.choices[0].message.content)
# "Nice to meet you, Alex!"

A basic LiteLLM completion call. The model responds to your message.

Now try asking the model about something from the previous call. Watch what happens when each call is independent.

01-agent-fundamentals.ipynb
python
# Second call: ask if it remembers
response = completion(
    model="gemini/gemini-2.0-flash-exp",
    messages=[{"role": "user", "content": "What is my name?"}]
)
print(response.choices[0].message.content)
# "I don't have access to your name..."

The second call has no idea about the first. Each call starts fresh.

No! That's exactly the point. Each completion() call is a fresh start. The model has no built-in way to carry information between calls. This is the amnesia problem, and it is the reason we need to build agents.

Quiz: Quiz

Loading practice…

Stateless LLM vs stateful agent

A stateless LLM treats each call independently. A stateful agent maintains conversation history across calls.

An agent is built on 4 pillars: 1. Brain, the LLM that reasons and generates responses 2. Memory, the conversation history stored in your code 3. Tools, external functions the agent can call 4. Planning, the logic that decides what to do next In this module, we focus on the first two: Brain and Memory.

The solution is surprisingly simple: keep a Python list of all messages and send the entire list with every API call. The LLM sees the full conversation each time, creating the illusion of memory.

Yes, and that is a real trade-off. Every turn costs more tokens because the history grows. We will tackle this later, with sliding window and summarization strategies. For now, the simple list approach is the right starting point.

01-agent-fundamentals.ipynb
python
# The key insight: maintain a list of messages
memory = [
    {"role": "user", "content": "My name is Alex."},
    {"role": "assistant", "content": "Nice to meet you, Alex!"},
    {"role": "user", "content": "What is my name?"}
]

# Send the FULL history every time
response = completion(model="gemini/gemini-2.0-flash-exp", messages=memory)
print(response.choices[0].message.content)
# "Your name is Alex!"

By sending the full conversation history, the LLM can "remember" previous turns.

Now you understand the problem and the solution concept. In the next lesson, we'll wrap this into a proper SimpleAgent class.