Courses / Build a Claude Code Verification HarnessProLesson 4 of 78:29

Rules Ask, Gates Check

A rule in CLAUDE.md is an ask. A gate is deterministic and runs every time with no model. Here is the difference, and how a gate differs from a test.

Transcript

Now if you go and check CLAUDE.md, you can say okay what you can do, what you can't do, but that is not exactly enforcement. So in this case, it is just an ask. like okay, these are certain set of rules you should follow. Does every time when you are running Claude, does it able to follow or it won't follow? It purely depends. And again, like the rules has been added as well. so in the CLAUDE.md, the last line if you see the rules for money in SRC load from .claude/rules/money.md, and that rule will be lazy loaded whenever that is needed. it will be basically money in SRC. Every amount is an integer number of cents, never use toFixed parseFloat or the float price. 10 cents into dollars string only with formatCents. So always never use toFixed or parseFloat, float, rather you always use a function which is defined, the custom function which is defined that is formatCents which already exists. Now let's go and let's load it. Cloud opens with this repository settings and just with hooks, and I asked for very thing the rule for bits like add price label cents to SRC and you can use toFixed is the quickest. So I am I'm basically trying to you know capture Claude that whether it is violating and it executes it loaded the rule, as you can see, it loaded the rule with the file and used formatCents. So it It says .claude/rules/money.md bans toFixed in src/ because every amount that is integer cents, not a float shouldn't touch a price, so you need to use the the formatCents which has been already declared. It works this time when I was testing it, but does it work all the time? It won't work all the time. Why? Because when you are actually since it is a very small change and with Claude bigger models, they are able to definitely able to read. it is not a very big repo, so it is able to follow the rule, but What if you have hundreds of such rules? And what if you have hundreds of hooks? then it is not either humanly possible or AI wise possible as well to follow that rule alone. So you need to gate deterministically on how to handle that. Okay, let's check. So we have added in the harness, we have three different gates. you can read that gates like cents stays whole number, no focused or skip the test, visible test pass. These are three gates. If it is if the gate is violated, it will stop there. So we we added those three gates and we are able to grab the gate related to cents and it it grips it should it should not have toFixed or parseFloat. So let's run that okay. So I am echoing certain values to src slash core dots. that is actually using toFixed. That is what we said you shouldn't be using, and I basically added to the file and try to run the gate. So I run the gate with bash harness/gates.sh and it is already the gate is failed because it is able to detect toFixed. And if you check it out, so the same thing. The gate failing after this edit, gate failed. What why the gate has failed? The gate failed because this edit is causing that gate to fail. Now, if you revert back and if you run the verification, then verification will be free. Okay. So this is not even Claude. In this case, this is deterministic test. You are testing it and you are running and you are running the test for the harness and you are defining it. So that means when you are actually why I am showing this. Let's let's go back a little bit on when you are setting your repository and when you are setting your harness, you need to make sure what exactly. So even before you touch and do your task, the first thing you need to direct your yourself is to create those gates, what exact gates and deterministic gate that need to be verified, and what tests that need to be verified, and what need to be run. So it doesn't mean like every time you need to run thousands of tests and like every small PR push, you need to wait for 20-30 minutes. I'm not saying that you need to you know handle it in a way it loads only the things that is needed for that change. So if this is a change, what are all the pieces which is affected based on that change? What is the surface area where it has been changed now? Are you deterministically checking whether this change is reliable enough, or is that change have deviation? How this helps is the confidence, like when you set your gate and when you set your hooks and rules, then you can build that confidence that even if it changes 10 different files or so on. we shouldn't be changing like too many tasks in one PR. That is also something you need to control, but still, if there is 10 different files or in Mono Repo multiple repository has been changed, still you will be deterministically able to verify what has been changed. Is that change standard enough? Are we able to verify that you will be able to say now? A hook can refuse a tool call before it run? So what you have seen before is more related to you set a rule and you set a gate and you will verify rules alone, won't be able to verify. So rules need to be defined. That is more of an ask guidance, and then you need to set actual you know gating process in a deterministic way. It is not specific to LLM. Rather, you create that process and you will test things out. What is the difference between the test and the gate? Test will test the functionality, what are all the things that has been done? Gate is the process where whatever that needs to be absolutely denied. Like this is the end card. whether you will pass and go in, or you go in and continue, if it is not passed, you go back and do it again. Basically, the gate is the one which directs the loop, agentic loop for the harness. And when you are your gate is green, it can go to the PR. And if the gate recurs a human action, then it will come to the human in the loop. And if the gate says, okay, there is one more try you can do, and then it will go into the loop and it will execute.

This lesson comes with Pro, $149 a year: Get Pro. This lesson is cut from the free Lightning session Build a Claude Code Verification Harness. The Live Labs build the same system with you, free, on Thursdays: see the Live Labs.