English
Book an intro call

Code review when AI agents write most of the code

When AI agents write the code, the bottleneck moves from writing to checking. A single developer can now produce in one morning what used to be a week of changes, and someone has to read all of it. Teams that keep reviewing everything the way they always did get stuck. Teams that skip review pay for it later, in production.

What the data shows

Google Cloud’s 2025 DORA report, based on about 5,000 professionals, found that instability in what gets delivered keeps rising as AI use grows. Data from companies that sell tools in this space points the same way, and should be read with that in mind:

  • In December 2025, CodeRabbit analyzed about 470 pull requests. Those written with AI averaged 10.83 issues, compared with 6.45 in pull requests written by people. Security issues were 1.57 times higher.
  • Faros AI, drawing on data from about 22,000 developers in the second quarter of 2026, found that in teams with high AI use, median review time was five times longer, pull requests were 51% larger, and production incidents per pull request were far higher.
  • According to Veracode, as of July 2026 only 56% of the code written by AI models passed security tests.

None of this argues for stopping the agents. It argues for building a review process around them that can keep up.

How Anthropic does it

On October 15, 2026, the Claude Code team at Anthropic published a detailed account of its review process. Claude writes about 90% of the code in Claude Code itself. In the company’s main repository, the number of changes landing each day grew from about 2,500 in May to more than 30,000 in September.

Here is how the process is set up:

  • Every pull request gets a review from Claude and a separate security review. An open finding blocks the merge.
  • An automatic classifier assigns each change a risk level. Low-risk changes merge with no human approval, medium-risk changes need one person, and high-risk changes need two.
  • Changes to the build and deployment infrastructure files, and to paths marked as sensitive, always go through a person. A bot’s approval never replaces a human’s.
  • The number of rounds is capped. If a change has not passed after three reviews, the team rethinks the approach instead of patching further.

The results, from June to September 2026: merged pull requests more than doubled, and the share of changes that were reverted fell by more than 40%. The share of pull requests that passed after one round of fixes rose from 65% to 78%, and the share that needed five or more rounds dropped from 8.9% to 1.5%.

Keep in mind that Anthropic sells the tools it describes. Even so, the structure itself, with risk levels and a person at the sensitive points, does not depend on any particular tool.

A check that runs beats a check that reads

A model that reads code and comments on it is one layer. The stronger layer is a test that runs and fails. Vercel’s code review agent is one example: it builds the fix it suggests, runs it in an isolated environment with the project’s build, tests and static analysis tools, and posts only the suggestions that passed.

OpenAI went further. In February 2026 a team there described five months of development in which not a single line was written by hand: about a million lines of code and about 1,500 pull requests. Dedicated linters and structural tests enforce the architecture, and agents review each other’s work. In that setup, human review is optional, unlike Anthropic’s approach. The gap between the two is exactly the question every team has to settle for itself: where a person has to be involved.

Boris Cherny, who created Claude Code, said in January 2026 that his most important tip is to give the agent a way to check its own work, and that this makes the result two to three times better. He also described a simple habit. Every mistake the agent makes gets written into the project’s instructions file, so it does not happen again.

Where to start

  • One repository, not the whole organization at once.
  • A short instructions file for the agent, plus a review rules file where every comment that keeps coming up in reviews gets moved.
  • An automatic review and a local security check before opening a pull request, then again automatically on every pull request.
  • Mandatory automated tests, with no merge without them.
  • Risk levels: what merges without a person, what needs one and what needs two. Infrastructure, permissions and payments always go to a person.
  • A template for every pull request covering how it was tested, what the risk is and how to roll it back.
  • One-step rollback, and flags that let you switch off a new capability without a deployment.
  • Every production incident becomes a new test.

According to Anthropic, it is worth spending a few weeks tuning the review rules in the first repository before expanding. This is also how we work with development teams: one team and real work, and only once the method is running do other teams join.

Frequently asked questions

If AI reviews code that AI wrote, isn’t it just checking itself?
That is a real risk, which is why the model’s review is only one layer. The other layers are tests that run and fail, a separate security review, and a person for medium and high-risk changes. At Anthropic, a bot’s approval never replaces a human’s.
How much does an automated review cost?
In March 2026, Anthropic cited an average cost of $15 to $25 per review with its tool. Compare that with an hour of a senior developer reading the same change, and with the production incident the review prevents.
Who decides what counts as a high-risk change?
At Anthropic, an automatic classifier decides based on the change itself, and sensitive paths have owners defined in advance. For a team that is just starting out, a manual list of sensitive folders and files is enough as a starting point.

Sources

Want to talk about this in the context of your system? Book a free one-hour intro call.