Your AI pipeline is fast. Your review gate isn’t.

Across the projects our engineers are working on right now, one request comes up more than any other: clients want to automate more of the software development lifecycle with AI, from code generation through to testing and deployment. In almost the same breath, they raise the same worry. If every AI-generated change still queues for a human to review it the old way, the review step becomes the new bottleneck, and the point of automating the pipeline quietly disappears.

That worry is well founded, and solvable, but not by choosing between two bad options: removing humans from the loop entirely, or reviewing everything by hand as before. The answer is to design the human-in-the-loop (HIL) checkpoint deliberately: tier review by risk, so higher-risk changes get more scrutiny and low-risk changes get less; run review asynchronously wherever the workflow allows it; and reserve human attention for genuine judgement calls rather than blanket approval gates that exist mostly out of habit. Get that design right and automation compounds. Get it wrong and the bottleneck simply moves three steps to the right.

This piece covers how to get it right, with reference to what current research and production systems show actually works.

Why “AI writes, human reviews everything” breaks down

The instinct to add a human checkpoint after AI-generated code is correct. The mistake is assuming the checkpoint can look the same as it did before AI changed the volume and shape of what arrives at it.

DORA’s 2025 State of AI-assisted Software Development research found that pull request size is up 51.3% year on year, a trend the report links directly to AI-assisted authoring, and names the resulting effect plainly: PR bloat continues to create cognitive overload and longer review cycles. The asymmetry is structural, not a training gap: an engineer using AI can generate in minutes what previously took hours, but the person reviewing that change still has to build the same mental model of what it does and why, at the same pace as before.

DORA also reports that higher AI adoption correlates with increases in both software delivery throughput and software delivery instability at the same time. That is not AI failing to help. It is AI amplifying whatever review discipline already existed. A team with a fast, well-scoped review process ships faster and more safely. A team whose review process was already a rubber stamp now rubber-stamps larger, riskier changes, faster.

The consequence engineers report to us directly is the one DORA’s qualitative data also surfaces: developers spend less time writing code and more time “babysitting the AI and reviewing what it is trying to do,” in the words of one developer quoted in DORA’s research. If every larger, more frequent pull request routes through the same manual, synchronous, all-or-nothing review gate that existed before AI, the queue in front of that gate grows faster than the team’s capacity to clear it. Automation upstream, congestion downstream. The fix is not fewer AI-generated changes; it is a checkpoint designed for the new volume and shape of work arriving at it.

The design principle: match scrutiny to risk, not to source

The naive HIL pattern applies the same level of human review to every change, regardless of what the change actually touches. A one-line copy fix and a change to the authentication flow get identical treatment because “a human should review AI output” was implemented as a blanket rule rather than a risk-calibrated one.

Risk-tiered review inverts that. It classifies changes before they reach a human, and routes only the ones that warrant judgement to a person, while letting genuinely low-risk changes proceed on the strength of automated checks alone.

Meta’s engineering team has published details on exactly this pattern at scale. Its Diff Risk Score (DRS) system uses a machine learning model, trained on historical linkages between diffs and the incidents that followed them, to estimate the probability that a given change will cause a production incident. That score feeds RADAR (Risk Aware Diff Auto Review), a pipeline combining the risk score with static heuristics, automated LLM-based review, and deterministic validation to decide whether a diff can be auto-landed or needs a human. Meta reports that diffs auto-landed through RADAR had a revert rate one-third that of diffs that were not, and a production incident rate one-fiftieth that of non-RADAR diffs. That is a case for risk-based automation, not against human review: the diffs sent to a human are exactly the ones where judgement was actually needed, and the ones that were not sent performed better because the routing logic was doing real classification work rather than defaulting to “everyone reviews everything.”

Smaller teams have found similar results without building an ML risk model from scratch. The startup Ona has published results from running an AI agent that auto-approves pull requests it classifies as low-risk while escalating everything else, with a human always performing the actual merge as the accountable, auditable action, reporting a 74% reduction in lead time. The details of the gating logic matter less than the shape of the pattern: classify first, escalate only what needs a person, and preserve a clear human action for anything that ships.

What “low risk” and “high risk” actually mean in practice

A workable risk taxonomy for SDLC automation typically separates changes along a few concrete axes rather than a single subjective “risk” label:

• Blast radius. Does the change touch a single, well-tested module, or does it cross a service boundary, touch shared infrastructure, or affect multiple consumers of an API?
• Reversibility. Can the change be rolled back cleanly (a feature flag flip, a container redeploy) or does it involve a data migration, schema change, or anything else that is hard to undo once it runs?
• Sensitivity of the domain. Does the change touch authentication, authorization, payment processing, PII handling, or anything with a compliance obligation attached, regardless of how small the diff looks?
• Test and eval coverage. Is the change fully exercised by an existing automated test suite and, where relevant, model evals, or does it land in code that is thinly tested?
• Provenance. Was the change generated end-to-end by an agent with no human edits, drafted by AI and substantially edited by an engineer, or written by a human from scratch? Each carries a different prior on risk.

A practical tiering scheme built on those axes looks roughly like this:

The point of the table is not the exact tier boundaries, which every organisation will draw differently depending on its risk appetite and regulatory obligations. The point is that the boundaries are explicit, written down, and enforced by tooling rather than left to individual reviewer judgement on a case-by-case basis. That is what turns “a human should review this” from a vague anxiety into an engineering control.

Asynchronous review: decoupling “reviewed” from “blocked”

Even within tiers that do require a human, the biggest single lever for avoiding bottlenecks is making review asynchronous wherever the change does not strictly require it to be synchronous.

The default pattern in most teams, AI-assisted or not, is that a pull request blocks merge until a human approves it. That default made sense when PR volume was low and review was cheap relative to the rate of change. It stops making sense once an agent can produce a dozen well-scoped, well-tested changes in the time a human reviewer can carefully read one.

An asynchronous pattern separates “this change is safe to progress” from “a human has signed off on it.” A Tier 2 change can merge to staging, run its full automated suite, and even reach a canary rollout before a human reviewer has looked at it, provided the rollback path is cheap and the blast radius is contained. Review still happens, but in parallel with progress rather than gating it. If the reviewer flags a problem, the rollback undoes the canary rather than a queue of blocked work.

This is a genuine shift in what “review” means: from a gate all work must pass through in sequence, to a control that runs alongside execution, with the power to halt or reverse rather than the requirement to precede. That shift only holds if two things are true: the change is genuinely reversible, and the automated checks that ran before the human looked at it are trustworthy enough that the human is reviewing intent and design rather than re-verifying correctness from scratch. Both are testable, tunable properties of a pipeline, not articles of faith.

What automated code review and evals take off the human’s desk

Risk tiering decides who reviews what. Automated code review and evals decide how much is left for that person to actually look at once a change reaches them.

The state of AI code review tooling has moved well past linting. Tools such as Qodo, CodeRabbit, and Graphite’s review agent now do context-aware analysis: summarising the intent of a change, flagging architectural inconsistencies, catching a meaningful share of real runtime bugs (industry benchmarking puts leading tools in the 42 to 48% range for real-world bug detection, ahead of traditional static analysis) and suggesting fixes inline. Run before a human opens the diff, these tools handle the mechanical part of review: style, obvious bugs, missing test coverage, security anti-patterns, dependency issues. What reaches the human, when this layer works, is a change already checked for what a machine checks well, leaving the reviewer free for what a machine checks badly: is this the right approach, does it fit the system’s actual constraints, does it introduce risk the static analysis cannot see.

Evals extend the same idea to behaviour rather than code structure. Where a team has a decent eval suite for a feature or agent, whether that is scenario tests for an LLM-backed feature or a regression suite for business logic, passing evals becomes a legitimate condition for auto-merge on lower-risk tiers, the same way passing unit tests already is. The question for a human reviewer shifts from “does this work” (the evals already answered that) to “should we want it to work this way,” a smaller, more genuinely human question.

None of this removes the reviewer. It removes the parts of the job that were never a good use of a senior engineer’s attention, and it is precisely what makes risk tiering safe: a Tier 1 auto-merge is only trustworthy because the automated checks it depends on are doing real work, not rubber-stamping.

Where deployment gates and rollback design fit in

Everything above concerns the checkpoint before a change merges. The other half of a well-designed automation pipeline is the checkpoint after it ships, because a good pre-merge risk assessment reduces the odds of a bad outcome, but it does not eliminate them, and a system that assumes it has eliminated them is not actually risk-tiered, it is just optimistic.

Progressive delivery patterns, canary releases, feature flags, and blue-green deployments give the pipeline a second, cheaper safety net that does not depend on catching every problem before merge. A canary release exposes a change to a small slice of traffic first, with automated monitoring watching service-level objectives; if error rates or latency breach a threshold, the rollout halts and reverts automatically rather than waiting for a human to notice. Tools like Argo Rollouts implement this by continuously querying metrics during rollout and triggering automatic rollback the moment an SLO is breached. Because the rollback is a traffic-weight change rather than a fresh deployment, it completes in seconds once triggered (Opsio, Argo Rollouts: Canary & Blue-Green Progressive Delivery), against the tens of minutes a manually diagnosed, manually reverted incident typically takes.

This matters for HIL design because it changes the cost-benefit calculation at the merge gate. If a bad Tier 2 change can be caught and reverted automatically within minutes, with no lasting customer-visible impact, the argument for blocking every Tier 2 merge on synchronous human sign-off weakens considerably. The deployment pipeline becomes part of the risk control, not just the thing that happens after risk control is done. Teams that design HIL checkpoints without accounting for what happens after merge tend to over-invest in pre-merge review, doing work a well-designed rollback mechanism could do more cheaply and just as reliably.

Claude Code as a concrete example of checkpointed autonomy

The pattern above, tiered risk, asynchronous review, machine-checked correctness before human judgement, is easiest to see concretely in how agentic coding tools are structured to let teams extend automation without losing control.

Claude Code, powered by Anthropic, is built around an explicit spec, plan, task, and implement loop rather than a single black-box “generate code” step. A developer or team defines the specification and the plan before the agent starts implementing, which creates a natural checkpoint: the human reviews and approves the plan, a cheap, fast, high-leverage judgement call, before the more expensive step of full implementation runs. That is risk tiering applied to the development process itself: the highest-value human intervention point is not the finished diff, it is the plan that determines what the diff would contain.

Once implementation is underway, Claude Code’s hooks let teams encode deterministic guardrails around what the agent can do, rather than relying on the agent choosing to run checks on its own initiative. A hook can run the test suite after every change, block a commit that fails linting, or require explicit confirmation before the agent executes a command against a production-adjacent environment. Because hooks run as shell commands rather than another layer of model judgement, they behave the same way every time, which is exactly the property a Tier 3 gate needs: predictable, auditable enforcement rather than another probabilistic decision layered on top of the first.

Claude Code’s checkpoint system, meanwhile, saves the code state before each change and allows an instant rewind through the /rewind command, decoupling “let the agent proceed” from “commit irreversibly to what it did.” That is the asynchronous review pattern again, expressed at the level of a single coding session: a developer can let an agent work ahead on a broader refactor, confident that a bad direction is a rewind away rather than a rebuild. Subagents extend the principle further, letting one agent delegate a scoped, lower-risk piece of work while a human’s attention stays on the higher-judgement part.

None of this is about letting an agent run unsupervised. It is the same principle as RADAR or an asynchronous review queue, applied at a finer grain: define the checkpoints deliberately, make the low-stakes ones cheap and fast, and reserve synchronous human attention for plan-level decisions and genuinely irreversible actions.

Regulated industries need sharper tiering, not more humans

Teams in financial services, healthcare, energy and utilities, telecommunications, and logistics often assume regulatory obligation means every AI-touched change needs a human in the loop, full stop. That is a reasonable instinct but not, on inspection, what the regulation usually requires, and it tends to produce the worst version of the bottleneck: a blanket gate applied uniformly across a codebase where only a fraction of changes carry compliance exposure.

What regulated environments generally need is not more human review overall, but sharper, better-documented risk tiering: a clearer, defensible line between changes that touch a regulated data flow, a controlled process, or an auditable decision, and changes that do not. A logging format change in a financial services platform does not need the same sign-off as a change to how a transaction is authorised, but under a blanket HIL policy it often gets exactly that, at the cost of review capacity the genuinely sensitive changes need. Getting the tiering criteria right, and documenting them so an auditor can see why one class of change was auto-mergeable and another was not, does more for compliance and velocity than adding reviewers to a queue that treats everything as equally sensitive.

Conclusion

AI did not create the need for human judgement in software delivery. It removed the excuse for applying that judgement everywhere at once. A checkpoint that treats a copy fix and an authentication change identically was never really risk management, it was habit dressed up as caution, and it becomes untenable once AI multiplies the volume of change arriving at the gate.

The fix is not fewer checkpoints or more of them. It is tiering by risk, running review asynchronously wherever the change allows it, and treating the deployment pipeline itself, canaries, feature flags, and automatic rollback included, as part of the risk control rather than something that happens after risk control is done. Get that design right, and a growing volume of AI-generated change becomes something a team can absorb rather than something it has to survive.

Where Zartis Fits

Getting this right is an engineering design problem before it is a tooling purchase. The teams we work with are not usually short of AI coding tools. They are short of a deliberate answer to which changes need a human, when, and why, and a pipeline that enforces that answer consistently rather than leaving it to individual judgement on any given pull request.

Zartis works with engineering leaders on both halves of that problem. The advisory work maps a codebase and its risk surface into a defensible tiering model. The delivery work builds the pipeline, hooks, evals, and deployment gates that make that model real rather than aspirational, including hands-on work with Claude Code, as a Preferred Services Partner in the Claude Partner Network, to design spec, plan, task, and implement workflows with the checkpoints a given team’s risk appetite actually calls for, from hook-enforced test gates through to rollback-aware deployment pipelines. The pipeline is built and run alongside the client’s engineers until it is theirs to operate, not handed over as a set of recommendations.

A faster code output paired with a slower review queue is a design problem with a known set of fixes. Tier the risk, automate the low-tier gate, and let the deployment pipeline carry more of the safety net than the merge gate does today.

Sources

• DORA, Balancing AI tensions: Moving from AI adoption to effective SDLC use, 2025 State of AI-assisted Software Development research
• Faros AI, DORA Report 2025 Key Takeaways: AI Impact on Dev Metrics
• Meta Engineering, Diff Risk Score: AI-driven risk-aware software development
• arXiv, Automating Low-Risk Code Review at Meta: RADAR, Risk Calibration, and Review Efficiency
• Ona, How auto-approving low-risk PRs with AI cut our lead time by 74%
• Anthropic, Enabling Claude Code to work more autonomously
• Claude Code Docs, Automate actions with hooks
• Qodo, Best Automated Code Review Tools for Enterprise Software Teams