Courses / Build a Claude Code Verification HarnessProLesson 6 of 79:14

The Stop Hook Closes the Loop

The agent says done. The Stop hook runs the gates, and red work goes back with the reason until it is green or a human steps in.

Transcript

Okay, now let's clean up the working directory. So what exactly happens? Claude says done and the stop hooks find you know that is some of the things which which is actually not done then with a reason. So it is not done because of failing test, or it is not done because of a gate or so on, and it will be pushed back to Claude again, and Claude will work towards it, fix it, and able to produce answer. So if if it goes maximum iteration or it is not able to go back, then it will come to you know needs a human. In this case, I can't take that decision because there is a permission issue that doesn't permit whether you you wanted to allow or not allow you you decide, and based on that I will proceed. Now every part we just saw on one ticket. I have created three tickets. so in this case you can create work T without the holdout test, and you can try it yourself on those tickets. So, what I did is like I go to a dedicated tree for ticket one and you know writing and checking the harness verdict in the log and executing it as you can see. So let's check okay. So execute ticket one. So this is the skill I talked about. Maybe I can quickly show that skill a little bit. again, like I haven't shown the directory. as I said, this is where the harness lives in. You can again, like this is very very simple harness very specific to this example, and this is the holdout test, which is basically the product owner's test and the test directory and the tickets. The tickets are like these three, and this is basically like you can execute it, you can you know do this entire demo. you can follow the readme. You can able to run each and every thing what I showcased in the demo. Okay, and the skills will be in the cloud directory. so all the hooks which I have shown, like God stop and everything, which is set over here, and the rules is defined over here, and the skills related to ticket is over here, like it's very simple rule, like you just do this, and that's pretty much it. You you call you call this and with the argument and so on, it will automatically pull that ticket. Okay, let's go back and see where we are. Okay, it comes to ticket one. Let's execute the ticket one then and the ticket one is done with RED, three failing test and no test code. So once again we have prompted and it is green from the names alone, and don't ask mode, never waited on me. Okay. Let me check. Yeah. The second time it is able to tan it to green because the holdout test, the initial issue was the holdout test have case insensitive thing need to be done. Then what happens is like we provide that feedback back to Claude, and Claude is able to execute that. So the first time it fails with error, the second time that error was stopped by the stop hook that hey, your holdout test is failing because of this reason, and that stop hook provided it back to Claude, and Claude is able to verify okay, these are the things, additional things I need to pass, and it was able to write that code and it was able to produce that result. And then I asked the sub agent reviewer like after the thing. In most cases, you might be doing review in your CI or something. I just created a small sub agent just to demonstrate within the terminal itself. Again, my suggestion in a real repository is always write the holdout test with a separate agent and create your sessions with your agent, and you know the specs need to be refined to into a plan with its own test, which is matched with this holdout test, and after that execute it and verify it against that test within that agent, and then have a dedicated verification layer that is both with human as well as your automated verification layer. So I always say keep it even driven rather than everything within one agent. So oftentimes AI assisted coding will work up to certain point, but AI native solution requires multiple event-driven work. So you have your agent to write the test, write the spec. PM can write it on their own, and they can direct AI to write the specific holdout test with each every scenario, and then from the spec to plan, and then from the plan to actual execution from the execution, the verification. So you can split out into multiple agents, but multiple agents talking to each other and doing it, or multiple agents will hand off, and then you you drive it as a workflow that is up to you based on your engineering process, based on your team's size and resources and how your existing engineering process work. You can always customize these workflows, depends on how you exactly work and your team exactly work. What is the dynamics of your team? Okay, let's go. that's pretty much in terms of you know guarding. you need to you need to have proper guards in place, then automatically your agent will be smarter enough to react based on what has been guarded, and based on that it is able to rectify. Now, this is for you to do. You can go and check it out. that as I said, there are three tickets. the first ticket is I did, and we can do the second one, and and then you can do the third one. So you can git clone and you can run Dr. Dot SH and you can you can run these comments and check whether it is red or green, what needs to be done. Rules just ask, gates check and hooks enforce. So that's how it follows. Rules will be just an ask. Hey, you follow, and Claude is like you need to grade some holdout test beyond Claude's own test. Beyond Claude's own reviewing, you need to have a separate agent. It can still be Claude. There are companies, there are guys, especially I know, especially staff engineers and principal engineers. What they are doing is like they will write their plan with Gemini. Gemini is very good in terms of teaching and so on. Like it can act like a researcher in terms of detailed writing, and then they will do pure coding in terms of cloud code. The cloud code is very, very good enough in terms of you know very the reasoning for cloud code, especially with Opus 5.5 for long running tasks without you know going behind or you know, hallucinating was almost perfect. so you can try it out. I think they released Sonat 5.5 today, today or yesterday. so I I haven't like I think for this demo I used son at 5.5, and as you see the output quality was quite good. And then they will verify with codecs or whatever process they have or code rabbit or some other process. That kind of setup will work very well. but of course if you are an individual engineer builder, in this case, you don't need to spend multiple different tools. Rather, you can have cloud with one process for you know writing your spec and from spec to plan and then actual execution and then you know verifying as a separate layer.

This lesson comes with Pro, $149 a year: Get Pro. This lesson is cut from the free Lightning session Build a Claude Code Verification Harness. The Live Labs build the same system with you, free, on Thursdays: see the Live Labs.