Code Review for Agent-Generated Work#

TL;DR#

Reviewing agent-generated code is genuinely not the same activity as reviewing human-generated code wearing a different hat: some of what made human review work (asking the author what they were thinking, reading effort and intent into messy code) disappears entirely, some of what made it costly (ego, defensiveness, limited re-review bandwidth) disappears too, and a distinct failure pattern - code that is fluent, well-formatted, and confidently wrong - becomes the dominant risk instead of the sloppy-looking mistakes human review is tuned to catch.

The idea#

Traditional code review evolved around a specific set of assumptions: the author had reasons for their choices, even bad ones, and you can ask them; the author's skill and care are visible in the code's roughness or polish, which is a genuine (if imperfect) signal of how carefully edge cases were considered; the author has finite patience for review rounds, which caps how many times you can reasonably ask for another pass; and the author's ego is a real cost to manage - blunt feedback lands differently depending on who wrote the code and how attached they are to it.

None of those assumptions hold for agent-generated code, and the ones that don't hold change what review needs to do, not just how it feels. There is no "what were you thinking" - an agent's output reflects what pattern in its training and context was most probable to complete the prompt, not a reasoned design decision you can interrogate and get a truthful answer from (asking an agent to explain its choice after the fact typically produces a plausible-sounding post-hoc rationalization, not a report of an actual decision process). Surface polish is decoupled from correctness in a way it rarely is for human code - a human who writes clean, well-formatted, consistently-named code has usually also thought reasonably carefully about the logic, because both come from the same underlying care; an agent produces uniformly fluent, well-formatted code regardless of whether the underlying logic is sound, because fluency and correctness are different capabilities that do not necessarily travel together. And there is no ego to manage and no finite patience to husband - you can request a full re-review after every single change, as many times as needed, for free, which changes the economics of review from "spend your scrutiny budget wisely across N passes" to "scrutinize as much as the risk warrants, since additional passes cost time but not social capital."

This lesson is not "review agent code more suspiciously" as a vague attitude adjustment. It is about naming the specific places where the old review heuristics stop working, so you can replace them with heuristics that actually catch what's different.

How it works#

What genuinely changes#

No author to interrogate. Human review often resolves ambiguity by asking: "why did you do it this way instead of the obvious alternative?" A good answer can turn a red flag into a non-issue (there was a constraint the reviewer didn't know about); a bad or absent answer confirms the concern. With an agent, that channel is unreliable - the agent will readily produce an answer if asked, but the answer is generated to sound plausible, not retrieved from an actual decision process, so it cannot be trusted the way a human's explanation can. Practically, this means review has to resolve ambiguity by reading the code and its context more carefully, or by testing the behavior directly, rather than by asking and trusting.

Surface quality no longer signals underlying care. A human reviewer's eye is trained on human failure signatures: messy indentation suggests a rushed job, inconsistent naming suggests low attention, and a reviewer's gut sense of "this looks careless" often correlates with "this probably has bugs." Agent-generated code defeats this heuristic by construction - it is fluent, consistently formatted, and confidently named whether or not the logic underneath is correct. This is the single most-cited failure pattern in current practitioner and research writing on the subject: code that is plausible-looking but wrong, which passes the kind of fast visual scan that catches human sloppiness, because there is no sloppiness to catch. Review has to shift weight from "does this look careful" (unreliable signal) to "does this actually do what it claims, checked against behavior, not appearance" (a reliable but more expensive check).

Re-review is free; human review capacity is not. Because an agent has no patience to exhaust and no ego to bruise, a reviewer can ask for a full new pass after every requested change, and can run the same automated checks (tests, linters, security scanners) after every iteration without any social cost. This means the bottleneck in agent-generated review has shifted almost entirely to human attention itself, not to the number of rounds available. Industry data on this is direct: teams with high AI adoption merge nearly twice as many pull requests, but PR review time rises by roughly 90 percent in the same period, and AI-assisted PRs run substantially larger (in one large-scale analysis, roughly 2.6x larger at the 75th percentile than unassisted PRs) - the volume and size of what needs reviewing has grown faster than the mechanism for reviewing it, precisely because generating a diff got fast while giving it real scrutiny did not.

Distinct, largely non-overlapping failure taxonomy. Human authors under time pressure and agents produce different bug shapes. Human mistakes tend to be visible-in-the-diff: a forgotten null check, an off-by-one, an inconsistent variable name reused from a copy-paste. Documented agent failure patterns are different in kind: dependency hallucination (confidently importing a package that does not exist, at a rate found to average close to 20 percent across models in one benchmark - an attack surface, since a malicious actor can publish a package under the hallucinated name); security-pattern stagnation (syntax correctness on generated code has climbed sharply since 2023, but security-relevant pass rates on the same benchmarks have stayed roughly flat in the 45-55 percent range, meaning newer, more fluent models are not automatically safer models); and self-confirming test suites (when the same agent generates both the implementation and its tests in one pass, the tests tend to encode the same assumptions as the implementation, so they pass even when those shared assumptions are wrong - a test suite that agrees with the code it is meant to be checking, rather than checking it against an independent specification). None of these three appear reliably on a human author's failure list, which is why importing human-review habits wholesale misses them.

What genuinely stays the same#

The actual review question does not change: does this code correctly implement the intended behavior, is it maintainable, does it introduce security or reliability risk, does it fit the codebase's existing patterns. The technical craft of review - reading a diff carefully, checking edge cases, running the tests, reasoning about what could go wrong under production load - is exactly as necessary as it was before. Nothing about the source of the code changes what "correct" means, and treating agent output as exempt from ordinary engineering scrutiny because "the AI probably got it right" is precisely the over-trust failure this lesson and agentic-software-engineering/06 are both about avoiding.

Worked example: the plausible-but-wrong pattern in practice#

An agent is asked to add rate limiting to an API endpoint. It produces a clean, well-named RateLimiter class, a decorator applied to the endpoint, and a passing test that calls the endpoint eleven times and asserts the eleventh call is rejected. On a fast visual scan, this looks correct and complete: sensible names, a test that exercises the boundary, no obviously sloppy code. Closer behavioral review reveals the limiter's counter is stored as an instance attribute on an object recreated on every request - meaning the rate limit resets to zero on every single request, and the passing test only worked because the test happened to reuse the same limiter instance across its eleven calls, a detail the test's own setup created and then silently relied on. Nothing about the code's surface - naming, formatting, structure - signaled this; a reviewer scanning for the human failure signature ("does this look rushed or sloppy") would find nothing, because it isn't rushed or sloppy, it is fluently wrong. Catching this required checking the actual claim ("does this limiter persist state across genuinely separate requests, the way production traffic will hit it") rather than trusting that clean code implies correct code.

Worked example: applying legacy-code skepticism to agent output#

software-engineering/legacy-code/01 defines legacy code precisely as code without tests - not old code, not code you dislike, and argues the risk that matters is the absence of a verified safety net, independent of who or what wrote it or how recently. This maps onto agent-generated code directly, with one important qualification. The parallel: code an agent just wrote, with no tests yet run against it and no track record in this specific codebase, carries exactly the same "I cannot yet verify this behaves correctly" risk profile Feathers describes for untested legacy code, regardless of how fluent or recent it is - fluency is not evidence of correctness any more than a codebase's age is evidence of risk. The qualification: unlike inherited legacy code, where the original author's intent is often lost and reconstructing it is expensive archaeology, agent-generated code was produced from an explicit, still-available spec or prompt (agentic-software-engineering/03) that states the intended behavior directly - so establishing what the code is supposed to do is typically cheaper for fresh agent output than for genuinely old legacy code, even though establishing whether it actually does that still requires the same disciplined, skeptical verification (tests, characterization of actual behavior, seams) the legacy-code subject teaches. Treat agent output with the same evidentiary standard as any code without a track record - "trust after verification," not "trust because it looks competent" - while recognizing you usually have better access to the intended spec than a legacy-code archaeologist does.

Pros#

Free, unlimited re-review rounds mean a reviewer can insist on real fixes rather than negotiating around limited patience, and can lean on automated re-checks (tests, linters, scanners) after every iteration at no social cost.
An explicit, available spec or prompt (when plan-then-execute and spec-driven development, agentic-software-engineering/03 and /04, are used) usually makes "what was this supposed to do" cheaper to establish than for genuinely orphaned legacy code.
Naming the plausible-but-wrong failure pattern explicitly gives reviewers a concrete thing to check for, rather than relying on a vague, unproductive sense that "AI code needs more scrutiny."

Cons#

The reviewer cannot resolve ambiguity by asking the author and trusting the answer, which removes a fast, often-reliable human-review shortcut and pushes more of the burden onto direct behavioral verification.
Surface fluency actively defeats the pattern-matching reviewers have spent years training on human code, which means naive application of old habits (skim for messiness, trust clean-looking code) is actively worse than doing nothing, not merely less effective.
Distinct failure modes (dependency hallucination, flat security pass rates despite rising fluency, self-confirming test suites) require specific checks a reviewer has to know to look for; a generic "review it carefully" instruction does not by itself surface any of them.
Rising PR volume and size without a matching rise in review capacity is a real, measured organizational cost (roughly 90 percent more review time under high AI adoption in one large study), not a hypothetical one.

Alternatives#

Treat agent output identically to human output, no adjustment — appropriate only for extremely low-stakes, easily-reversible changes where the cost of a missed plausible-but-wrong bug is negligible; for anything with real blast radius, this ignores documented, measurable differences in failure patterns.
Formal verification and property-based testing in place of (not alongside) human review — catches classes of error human skimming reliably misses (the self-confirming-test-suite pattern in particular resists any review approach that trusts the agent's own tests), but is expensive to set up and does not substitute for judgment about whether the intended behavior itself is right.
software-engineering/legacy-code's full toolkit (seams, characterization tests) applied to fresh agent output — a genuine option when agent-generated code is large, unfamiliar, and about to be modified further; more rigorous than a standard review pass, and worth the extra cost specifically when the code in question is about to be changed again, not merely read once.

When to use it#

Apply agent-aware review practices - direct behavioral verification over visual scanning, explicit checks for dependency hallucination and security-pattern gaps, skepticism toward agent-generated tests as an independent check - whenever the code under review was produced primarily by an agent, especially for anything touching security-sensitive paths, data persistence, or logic a human has not yet independently exercised. The higher the blast radius of being wrong, the more the plausible-but-wrong risk matters, since that is precisely the failure mode ordinary skimming will not catch.

When NOT to use it#

Do not apply the same heavyweight, behavior-first scrutiny to trivial, easily-reversible, low-blast-radius agent output (a config value tweak, a comment fix) where the cost of thorough verification exceeds the cost of being wrong - proportional review still applies, per agentic-software-engineering/02's controlled-use framing; agent-authored does not automatically mean high-risk. Equally, do not substitute the specific checks in this lesson for ordinary engineering judgment - naming what's different about agent review does not mean everything else about good review (readability, fit with existing patterns, actual correctness) stops mattering.

Key takeaways / mental model#

Three things genuinely change when the author is an agent: you cannot interrogate intent and trust the answer, surface fluency no longer correlates with underlying correctness the way it does for human code, and re-review becomes economically free, shifting the bottleneck to human attention itself. The single highest-value habit shift is moving from "does this look careful" to "does this actually do what it claims, verified against behavior" - because plausible-but-wrong is the dominant failure shape agent output produces, and it is specifically the shape that defeats a fast visual scan. Treat fresh agent output the way software-engineering/legacy-code treats any code without a track record: skepticism until verified, not confidence because it reads well.

Self-check questions#

A reviewer says "I asked the agent why it implemented the retry logic this way, and it gave a reasonable-sounding explanation, so I approved it." What is wrong with treating that explanation the way you would treat a human colleague's explanation, and what should the reviewer have done instead?
Using the rate-limiter worked example, explain why a reviewer trained on human code's failure signatures would be more likely, not less, to approve this specific bug without catching it.
software-engineering/legacy-code/01 argues legacy status depends on the presence of tests, not on age. Explain the one place this lesson says the analogy to fresh agent-generated code is not exact, and why that difference matters for how much review effort each requires.
Your team's agent-generated PRs are passing all tests and getting approved faster than human PRs, but production incidents attributable to those PRs have risen. Using the concepts in this lesson, what are two specific, different explanations for this pattern, and what would you check to distinguish between them?
A colleague argues that since re-review is "free" with an agent, the right move is always to request another full pass rather than fix a small issue yourself inline. Steelman this position, then explain when the "free re-review" framing breaks down in practice.

References#

Apiiro, 2025 Fortune 50 AI code security analysis (2025) - referenced in industry reporting on privilege-escalation and architectural flaw rates in AI-generated code
Veracode, 2025 GenAI Code Security Report (2025) - XSS and log-injection insecure-generation rates
LinearB, "8 million pull requests reveal where engineering productivity breaks down" / 2026 Software Engineering Benchmarks Report (2026) - https://linearb.io/blog/8-million-prs-engineering-productivity
Augment Code, "Reviewing AI-Generated Code: A Verification Discipline for the Loop" (2026) - https://www.augmentcode.com/guides/reviewing-ai-generated-code
Addy Osmani, "Code Review in the Age of AI" (2025) - https://addyo.substack.com/p/code-review-in-the-age-of-ai
Stack Overflow, 2025 Developer Survey (2025) - trust-in-AI-output figures
Working Effectively with Legacy Code (Michael C. Feathers), Chapter 1: "Changing Software" - see software-engineering/legacy-code/01