From a production error to a tested lesson
How RepoOps follows code an agent wrote from the error back to the session, gets a person to approve the fix, and measures it before learning from it.
Most teams now ship code that AI agents write faster than anyone can review it. When production breaks, four questions follow. Which change caused it, and which agent session wrote it? Who approved the fix? Did the fix work? How do we stop it happening again?
We built one loop that answers all four, in RepoOps, with PromptCompute for the approval. This post walks it step by step and marks what is built and what is next.
1. The error
An error arrives from Sentry, from browser reports, or as a route that got slower. A source map points at the file. When there is no stack trace, the route that failed can still name one.
2. The deploy and the change
RepoOps records which deploy served the error and the history before it, including changes that never touch git: feature flags, migrations, Railway changes, Dependabot and provider status pages. Then it names the commit. It has nine ways to do that, and we measure each one by replaying past cases with only the evidence that existed when the case opened.
3. The agent session
A Session-Id trailer on the commit links it to the session exactly. With the RepoOps commit hook installed, commits from Claude Code get the trailer automatically. Other tools need a session id set for them, or link through time and file overlap, which gives a strong or weak link. When two sessions overlap, RepoOps does not pick one, and the link reads weak.
The task also records the context the agent was shown, as a hashed packet. We will show whether the agent followed a rule as its own signal, apart from accuracy.
4. The approval
A rollback or a fix goes to a person, in the RepoOps approval queue or, for a team connected to PromptCompute, through PromptCompute. Sending the change risk to PromptCompute is built and not switched on yet. A denial that says the named change was not the cause becomes a label that corrects attribution.
5. The recovery
Every task can carry success criteria. Each one reads met, not met or not measured. Then RepoOps watches for 14 days. The fix reads as worked only if every criterion is met and the same cause has not come back. That is an observation, not proof the fix caused it. The 30-day rework rate sits beside it. A criterion we cannot measure says not measured, never a quiet pass.
6. The lesson
A bad outcome starts a lesson candidate, and a person reviews it before it reaches the blocks in CLAUDE.md and AGENTS.md that the next agent reads before it edits that code. RepoOps replays it against that repo’s past failures and shows the result, and a lesson needs that replay before it can go to the other products.
What is next
The task record that ties this together is built, with an append-only list of events. Next we will put the whole story on one timeline page, with each link marked measured, inferred or missing, and let a new customer replay a known incident read-only before granting any write access.
