From Pull Request to Merge
Every run ends in a pull request that waits for you. Log each run's decisions so the next one starts smarter, choose which stages run unattended, and decide between a light factory with humans at each gate and a dark one.
Chapters
Transcript
Decision logs
Now, every run ends with a pull request, and pull request will come to you and you become the one who holds it, and you are the one who is actually going to merge the pull request. So that is what I have mentioned. Like when you are actually having a huge diff, then automatically you will become the bottleneck. So your your team is able to generate a lot of code and a lot of things, but then it will still be slower in terms of deployment because you are not going to merge it until you get final satisfaction. So you will start to do manual testing, you will start to do everything just to make sure you are satisfied. so automatically, that is where the bottleneck happens. Now, in terms of data, whatever we are running, at that point itself, every code agent have a lot of knowledge, a lot of research, lot while executing itself, we will get a lot of details that we need to write it back to our wiki. So whenever a coding agent finishes running, what are all the knowledge that can be run? So this is many of the companies call it as decision logging. Like there are standard practices and standard templates which is available. So all the things, whatever we have discussed in the software factory design is basically powered through standard design, right? So in a way, a linear task or a click up task, you always have a standard template. The product managers use a standard template. And in GitHub, when you are creating you know, creating an issue, you will follow an issue ticket, how the issue needs to be, and so on. And when you are creating a PR, then also you will follow a standard template. Okay, what problem it is solving, how it is solving, what are all the landings you have, any screenshot that need to be there, what are all the tests you have done, and so on. Every company has a different way. The same way we need to maintain context. in terms of every engineer is running the Claude code session or VM is running Claude code session, where a lot of knowledge which happens. So each run a decision log that needs to happen, and that decision log needs to be managed automatically through the agent. That way, that way you will be able to evaluate how the software is actually evolving and how that decision is actually going to help you in your next you know coding session. So automatically, every time there is a next PR which is going to create it, that will become much more intelligent because there are a lot of other existing decision logging which will provide context to that.
What runs unattended
Now, specs approved, plan approved. checks are passed, merge waits for you. So it is like in a typical environment. you you you you create all the things and it creates a work tree and it creates build and test run, everything has been created and PR has been created, and then it will come to you. Which stage would you let run unattended on your repository this morning? Like now you might be already doing a lot of these automation probably either on an individual level or in your companies itself, you might be already automating, which are all the states you are more confident in your company that you can let run unattended on your repository. and nine out nine out of ten times, I am pretty sure this PR is good enough to go go live. that is what the software factory can create.
Light and dark factories
Now I think this is one of the example from my software factory. Let me go into the next one. So this is following an example, like how the review become bottleneck. So when it comes to verification, what we are talking about, like you become the bottleneck. So how you can take yourself out of it is only through you know granular planning when you are actually granularly planning it and automating as much verification as you want through a separate agent, then you will be able to handle that, and you will be you know you will be confident enough what are all the things which can be auto merged and what are all the things which can't be auto-merged. So, even in software factory, they call it as light factory and dark factory. Addy Osmani, probably you might have listened to his talk related to it. Light factory is more like you will still guide those you know human layer, like the spec to plan will be created, and then you will approve the plan, and then after the plan, it will come to the next stages, it will execute and it will it will create multiple PRs, and then you will start to you know provide the code review. Once again, it will go into execution stage and it will fix it and then you will merge it. As I said in my earlier slide, at the main thing, what we are automating is also mainly small bugs and Dependabot alerts and the mainly the pull request what is that pull pull request verification? So these are the main thing we are automating it when it comes to larger epics, then certain pieces are automated, but it is not fully automated to you know entire PR route. Rather, it will be like from the spec, create the plan. You have the context, so create the plan, but still that plan will be only creating a draft PR without any file changes. It will be manually you know gone through and discussed with the product manager and engineer, and then once they are satisfied with the plan, and then they will go into the next execution stage. Okay.
This lesson is cut from the free Lightning session Design a Software Factory for Your Coding Agents. The Live Labs build the same system with you, free, on Thursdays: see the Live Labs.