Self-correction with reflection

Here is the key insight: when the LLM returns invalid output, do not just fail. Feed the specific error message back to the LLM and let it try again. LLMs are excellent at self-correction when given clear feedback.

07-error-handling.ipynb
python
def smart_command_agent(user_query, max_retries=3):
    messages = [
        {"role": "system", "content":
            "Extract 'room' and 'temp' into JSON. "
            "Example: {'room': 'kitchen', 'temp': 22}"},
        {"role": "user", "content": user_query}
    ]

    for i in range(max_retries):
        response = completion(
            model=DEFAULT_MODEL,
            messages=messages,
            response_format={"type": "json_object"}
        )
        data = json.loads(
            response.choices[0].message.content
        )

        error = validate_command(data)

        if error:
            # FEEDBACK LOOP: tell LLM what went wrong
            messages.append(
                {"role": "assistant",
                 "content": json.dumps(data)}
            )
            messages.append(
                {"role": "user",
                 "content": f"Error: {error}. Fix and retry."}
            )
        else:
            return data  # Valid!

    return "FAIL: Could not fix command."

The reflection loop: try > validate > if error, tell LLM what went wrong > retry. Up to max_retries.

Self-correction cycle

07-error-handling.ipynb
python
# Example: LLM corrects itself
result = smart_command_agent(
    "make the kithcen 22 degrees"
)
# Attempt 1: {"room": "kithcen", "temp": 22}
#   Error: Room 'kithcen' not supported.
# Attempt 2: {"room": "kitchen", "temp": 22}
#   Valid! Return data.

The LLM reads the error message and corrects "kithcen" to "kitchen" on the next attempt.

AI prompt: Try it with AI

Loading practice…

Matching exercise: Match error types to correction strategies

Loading practice…

Quiz: Quiz

Loading practice…