Done Is Decided by a Check, Not the Agent
What decides that an AI agent's task is done? An LLM can say it finished; a deterministic check decides whether it did. In the demo, done is decided by GitHub showing the reply. After this lesson you can control code at the source with skills and formatting checks instead of line-by-line review, and use a failed check to send the agent back.
What this covers: done decided by GitHub, not the agent; the advice Param gives startup engineers who ask how to keep standards without reading every diff: define the standard in skills and formatting checks, so the system flags a file that is too long or not formatted; reviewing the spec, acceptance criteria and before and after screenshots instead of code pieces; the loop a check drives, where a failed check returns the log to the agent under a max turn count; checking the reply came from an authorized person; and why a prompt alone is not enough.
#aiagents #claudecode
Chapters
Transcript
Done is decided by GitHub
So now done is decided by GitHub, not by the agent. So, this is also as I said, whether this task is done or not, it shouldn't be you know said by LLM or it shouldn't be said by the AI agent which is working on top of the LLM. Rather, your deterministic process need to check okay, it says done whether this is done or not. This is not only for this issue, oftentimes this is the main thing I ask while doing my consultation for startup teams, their engineers they will come and ask me, like Param, the biggest issue is like we have our own standards in terms of coding practices, but now without verification line by line, how do I make sure that code base is not deviating? So, of course, as an engineer they have their own dilemma. So, here my advice is always like you know, the review at this point with AI agents. you you shouldn't be going after you know what kind of code changes that happened.
Control code at the source
You need to control that, not in the review process, you need to control that in the source level. When you are actually defining your skill, you need to define what is the standard way we operate, and in your formatting checks, you need to define what is allowed, what is not allowed. So, the system need to verify okay, this this file have more than 120 lines of code. This file is not formatted. The system needs to tell that rather than a person who goes and verify the code and says what is there? The person needs to just verify what is the spec which has been created with the AI assistance, and what is the approval acceptance criteria which has been created, and what is the before and after you know, screenshot if it is an UI related visual related task, and you just need to verify that whether this is actually in line with it, not the code pieces. The code pieces need to be verified automatically through your logic and through your gating process. This is mindset-wise, it is indeed difficult to grasp because until now we have been doing in a human way on a human scale, but when coding agents and like AI agents are becoming smarter and smarter, how you need you need to evaluate itself is the actual process nowadays. You know, building your evaluation for your either for your coding agent or for your actual agentic AI application is the actual thing, what product managers as well as engineers are currently doing, everything else can be automated and you just need to have what to review that you need to classify what AI will do, what human will do, and then you will be able to automate those processes.
The loop a check drives
The same here as well, done can be said by LLM, like I finished it. Now you need to deterministically verify whether the task is actually done or not. If you are able to verify, then you will be able to control the agentic loop. For example, the first time agent says done, and your deterministic check says it is not done, then it is the actual agentic loop you will once again trigger. Like, okay, my check says this this is the log, and go back to your AI agent, and you have created max turns, number number of max turns will be three, for example. Then the second time it runs, it has the informed information that okay, my done have some you know gaps. I need to fix the gap. Then it will you know do the reasoning and it will once again process that and it will make sure that done is justified. Okay, let's go. and then here you need to verify whether the actual reply is done by the person who is who is authorized to do or it is not done. This is standard process whether it is AI or not AI, we need to make sure you know our system is secure enough. so in an AI agentic application, as you might have seen news all over, like PHP got couple of large vulnerability, and many of the companies, like I think open code also have some of the vulnerabilities which has been exploited. So the more and more you need to think about you know AI evolves as well as you need to think about secure practices, regardless of whether it is an internal app or external app, you need to make sure the AI agent what it is producing, you need to deterministically verify whether the right person is doing that or not. Now we checked it, who replied, and that's it. So in this application, if you if you run these check, so I have written these tests to verify. So there are multiple verification check, you can read about it in the code base I shared, and you can run that as well.
The takeaway
So as you see, the prompt alone won't give you the confidence that this is enough. you need to make sure whenever you are creating your mission control or your team of AI agents, you need to make sure have a deterministic check in place, then you can actually verify that independently with the manager kind of agent, then you will be safely say, okay, this task is actually done or not. that you need to, it is not about just mission control for any AI agent, that is one of the most important pieces that you need to add that helps you to you know make a self-improving loop as well. Later on, when you are adding self-improvement based on what it runs before, you have the data, and then you can make calculative adjustment in terms of how it can improve on its own.
This lesson comes with Pro, $149 a year: Get Pro. This lesson is cut from the free Lightning session Build a Mission Control for AI Agents. The Live Labs build the same system with you, free, on Thursdays: see the Live Labs.