Verification Is the Bottleneck
The six layers of a software factory, and the one teams stall on. An agent returns a 120-file pull request marked verified; nobody is confident enough to merge it. Plan in smaller chunks so each change can be checked and shipped.
Chapters
Transcript
Six layers
These are the six layers that you need to define for your software practices, software process, then you can able to create that process for the factory. What are all those? Boundary, execution, context, skills, verification, and delivery.
The 120-file pull request
Oftentimes, the biggest bottleneck for everyone will be the verification. For example, if you create a spec and you ask Claude Code to take that spec and execute it, and you provide enough context that this is the repository and you provide skills, this is how I execute and everything. Now it creates a gigantic PR with 120 files. Now, who is going to verify what it has produced? Now it says I have verified this, this, this, and everything is tick. Now you can merge it. Are you confident enough to merge it? Is your engineering manager or your head of engineering is confident enough to merge that? That is where the gap is. Majority of the time, the execution can easily happen either in your local harness or in your factory in a VM system or in CICD or in a separate sandbox. You can execute that automatically, but it will come to a choking point where the verification becomes a bottleneck. When you are actually providing just a request, then it will produce, you know, certain result. This is my rules, this is my context, and these are all the checks you need to do. These are the tests you need to run, and these are the tools you can call, and automatically it will be able to create. But you still hold what are all the things that need to be done? like you are the one who is going to verify. So, even if the factory is able to generate like multiple tasks and multiple PRs at the same time, you are the one who is going to hold it.
Plan in smaller chunks
Then what you need to do in the planning stage itself, you need to make sure it is able to plan in a smaller chunk. Like this is an epic, and for that epic, what are all the task lists that need to be planned? And for that task list, what are all the things we can able to execute and then so on. We will be able to verify more smaller chunks, even if you have a human verification layer in place when there are smaller changes with clear details on what is changed, one file, two file kind of changes with clear depth, then you will be able to verify faster and you will be confident enough to merge and you can deploy it faster. Now, what is the advantage of a factory compared to compared to running on your own? Like, I can also open multiple terminals and I can feed, like okay, you do this task, you do this task, you do this task, and everything will be done. Now, the main challenges will be like you will do everything in one layer, but you need to set the process, like okay, create work tree for each of the tasks, and then you know, testing those in your local for each work T and everything will become much more complex. Then automatically, your brain will also like maybe four or five tabs you are able to manage after that. You have your own fatigue tool where I typed what or where I shared what it becomes much much harder for you to manage. So that is where you can do that factory more in an isolated scope, then you will be able to do much easier. Verification becomes the bottleneck, so you need to make sure the verification layer is thin enough so you can easily and confidently verify and merge it. Of course, for there are many one man startup and small startups who who does the verification automated as well with a with a risk portfolio, like for risk which is higher enough, then you know you don't merge it for risk which is lower enough, so you can get it that way as well.
This lesson is cut from the free Lightning session Design a Software Factory for Your Coding Agents. The Live Labs build the same system with you, free, on Thursdays: see the Live Labs.