Two discussions from last year point at an attention bottleneck that better model output alone doesn’t remove. In a thread from late last year about why agents still don’t write most production code, one engineer described watching the code as the agent changed it, ready to interrupt “if it is doing something stupid.” A few comments down, another was looking at their own backlog of a dozen pull requests waiting on review. A separate thread from the middle of last year asked whether people really run multiple agents at once, and one reply named the cost: “I have to watch what happens all the time, correct him and point him in the right direction.”
Those quotes point at two different costs. One is supervision while the agent runs: someone watching the work and correcting it in real time. The other is review after the agent stops, finished changes stacking up in a queue with nobody free to approve them.
Review is still one person at a time
Agents now produce candidate work in parallel, all night, and nobody added hours to the person who reviews it. Review is one human reading one thing at a time, same as it was.
Augment Code ran headfirst into this in January. They called it a code review wall: 1,400 pull requests open at once and a 20-hour median wait for the first human comment. GitHub, in its own guide to reviewing agent pull requests, says agent output is “already saturating review bandwidth.”
Without explicit triage rules, the queue presents work in arrival order instead of the order where your judgment matters. The change that touches billing sits next to the one that renames a variable, and an incident summary that needs a real call looks no different from a routine note. To know which items are routine, you have to review them, and that review is the cost you were trying to avoid.
The math is worth doing on your own setup. Multiply the number of outputs your agents produce in a day by your honest average review time. Twenty outputs at fifteen minutes each is five hours of judgment, every day the fleet holds that pace. More agents mean more outputs waiting on the same five hours.
A lot of teams read that math and split into two shifts: people review by day, agents run by night. It clears the morning queue, but it forces building and judging to take turns. The agents wait on you, then you wait on them. The work only ever moves in one lane at a time.
The better shape runs both lanes at once. That works when the one decision that needs you sits at the top of the pile, and everything that can run without you already is. An agent earns its way out of your loop by clearing verification that stands on its own: tests that pass and judged checks that still hold when you look.
The queue outlasts the model
Those threads are from last year, and part of what they describe has already eased. Flagship models need less steering than they did, and the frontier coding harnesses now handle a lot of the real-time correcting those engineers were doing by hand. How much steering an agent still needs mostly reflects how well it’s set up to succeed.
Better models cut the supervision during the work, and they finish more of it. Unless you change what requires a human’s sign-off, that extra output lands in the same review queue. That is a job for machinery you set up once.
Smaller and open-source models are a different story: they need more guardrails and more steering to run safely. Writing that steering down pays off twice. You stop re-deriving it on every run, and you stop depending on one lab to keep your setup working. Chaining a strong model where it earns its cost and a cheaper one elsewhere keeps any single lab from deciding what you can run. Whether it also gets you better results depends on the task and how you judge the output.
Review by exception
Review by exception means a person only opens the outputs that need a decision. Everything else clears on criteria you wrote, with the verdict recorded. The machinery has three jobs:
- Judged gates apply the criteria you wrote to every output and record the verdict before a person is involved. Whatever passes proceeds.
- Held gates wait for a person no matter how clean the output looks. You decide in advance which actions carry enough consequence to sit here.
- The attention inbox collects what’s left. Each item carries the evidence to decide on it: what the run did, what it spent, and what it’s waiting to do.
Every decision lands in the run’s receipt next to the person who made it, so “who approved this” still has an answer months later.
Other teams already built this
Parts of this shape already show up in three in-house systems we’ve written about. Cloudflare’s code review panel runs up to seven reviewers on a change, then a coordinator model dedupes their findings and posts a single review ranked by severity. A firehose of raw warnings, in their telling, is one developers learn to ignore. Ramp’s agent Inspect drafts a pull request, then gathers the evidence a reviewer needs: before-and-after screenshots and the checks an engineer would run locally. The person at the merge decides instead of digging. Stripe’s minions run the fixed checks every time and hand a human the merge, which is where they say the trust came from.
We run the shape ourselves, too. When our seven-model review panel split on a change, nothing paged anyone. The run parked in the inbox with the disagreement laid out, and the call took minutes because the evidence was already sitting there.
What makes a good gate criterion?
What if a judged gate passes something it should have caught?
How is an attention inbox different from notifications?
How the attention inbox works
→The concept in full, plus the gate mechanics it sits on.