Courses / Build a Mission Control for AI AgentsProLesson 5 of 54:45

Cap AI Agent Loops With Cost and Max Turns

How do you cap AI agent loops with cost and max turns? Max turns is how many times an agent may loop on a task; a cost cap is what a run may spend. Param prefers to set both as an agent's boundary. After this lesson you can set those two limits and track cost per outcome, the number to watch on each release.

Cap AI Agent Loops With Cost and Max TurnsPro lesson4:45

What this covers: why cost needs the same attention as evaluation; an AI SRE that ran fine on a Claude frontier model and kept looping on a cheaper model in a controlled run, and what the same loop could cost overnight in production; why a non-vision model given a vision job fails however many loops it gets; and cost per agent and per task, given Anthropic's finding that multi-agent systems use about 15x more tokens than chats. The terminal on screen illustrates the loop; it is not output from the real run.

#aiagents #llmcost

Transcript

Cost and max turns

So, usually this is outside of mission control per se, but usually the boundary I personally prefer to set or with cost and maximum number of times. When you didn't say what is the cost, you know or you you are not calculating what is the cost per run, then you won't know what is the output which has happened and whether we are actually saving or whether we whether we are actually wasting our token, we can't able to keep up. How important an evaluation is the same level of importance need to be provided for cost, then only you will be able to set your actual boundaries. Okay, I said maximum number of runs, you know, maximum number of agentic loop you can do is three. but you know, some team can do 10, some team can do 20. What if I didn't do anything?

A loop that burns money

then the agent will loop and loop and loop. This has happened to me actually. I have built an AI SRE for a for a company, and they are using ClickStack basically ClickStack in the sense ClickHouse and thing, and I have you know did a great workshop for their team, and I demonstrated okay, this is how the AI SRE will do the investigation and it will find the git logs and it will check okay, what is the what is the you know changes which has which has caused that issue, and I showed the demo and they deployed it and it was running like on the first run. they they were trying it was running multiple loops, and luckily we you know that is more like a calculated run. We are trying and running, and of course, they they used a very like cheapest model to run and verify how it works. I was running with Claude you know frontier model, and everything was seamless, and they were using DeepSeek v4 flash from OpenRouter, and it was looping and looping, and it was able to it was not able to resolve and it was not able to find because of the reasoning capability of the model as well as how it was done more specific for Claude agents, Claude LLM. So then we found out like that's it's not a very costly mistake because it was a more controlled run. But what if it is not a controlled run after the workshop? if someone is running it in production and overnight it was looping and it was you know burning money like hundreds or thousands of dollars. So that is where cost calculation and actual max budget, like when to stop, you always need to provide a criteria of gating. Gating process need to be there in terms of maximum iteration, stop it. Then a human will come and say what is going wrong and what needs to be fixed, whether it is in the AI agent or whether it is in it is in the loop logic, or whether this run or this model is not sufficient enough, then you need to pick and choose the right model for right job and so on. For example, if you choose a non-vision model for a vision job, then most probably if you are having an having a loop of a hundred times, still it is going to fail because it is going to hallucinate and say, Okay, this is what I read, even though they might not even able to read, they will just say based on the topic you provided, it will say I read this PDF and I found all the things and I was able to provide answer based on the PDF, and it is just hallucination. So that is very important. You need to define the exact boundaries, not just on scope level, but also on you know, globally, what is the budget and what it needs to run and how many times it needs to run and so on.

Cost per outcome

So, Anthropic have said like 15x more token than a chat for a multi-agent system. So, cost per outcome is the number to watch. You need to watch what is the cost per agent and cost per task that you need to calculate, and every time when you are actually releasing, you need to make sure not just you need to evaluate, but you need to make sure your cost per task is going up or down and how it is calculated.

This lesson comes with Pro, $149 a year: Get Pro. This lesson is cut from the free Lightning session Build a Mission Control for AI Agents. The Live Labs build the same system with you, free, on Thursdays: see the Live Labs.

That was the last lesson. Pro members watch every Lightning session cut the same way: Get Pro. Not sure what to learn next? Take the 2-minute assessment.

Chapters

Transcript

Cost and max turns

A loop that burns money

Cost per outcome

Build it live, with Param reviewing your choices