Seven models reviewed the same change. They disagreed.

Last week we changed the retry logic in our own scheduler and put the merge request through the same review playbook we run on every change: seven models read the diff, each with the same instructions, each writing its review before any of them sees another’s.

That last clause matters. If reviewers can see each other, they converge. The second model anchors on the first, the third summarises the first two, and by the end you have one opinion wearing seven names. So the playbook runs the reads as isolated stages. Nothing crosses between them until every review is filed.

Here’s roughly what came back on a 400-line diff:

• claude-opus-4-7 flagged that a failed cleanup inside the backoff path was caught and logged but never rethrown, so a run could report success after a partial teardown.
• gpt-5.3-codex read the same lines and called them fine. It spent its review on a query that would scan a whole table once retry counts got large.
• kimi-k2.5 and deepseek-v4 both landed on the swallowed error too, independently, in different words.
• gemini-3-pro questioned whether the new jitter window could starve the queue under load. Nobody else raised it.
• qwen3-235b and glm-5 filed short, clean reviews: style notes, one naming complaint, no blockers.

Four of seven converged on the swallowed error. One raised a load concern no other model saw. Two saw nothing serious.

Disagreement is the product

When four models from three different labs independently point at the same line, you fix that line. No debate needed. That’s consensus doing your triage.

The lone dissent is the more interesting case. gemini-3-pro’s starvation concern was the kind of thing that’s either nothing or an incident. It cost us ten minutes to check (it was nothing, the window is bounded upstream), but the point is that the panel put a bounded, checkable question in front of a human instead of a vague feeling that something might be off.

And the two quiet reviews are signal too. If every model files blockers on every diff, your panel is noise. When the quiet ones stay quiet on routine changes and light up on real ones, you learn to trust the distribution.

Why the panel crosses model families

A panel drawn from one lab fails together. Models trained in the same house share blind spots, so their agreement tells you less than it appears to.

Sumn treats open-source models as first-class panelists: you bring your own keys, point a stage at any OpenAI-compatible endpoint, and kimi-k2.5 sits in the same DAG as claude-opus-4-7 with the same instructions and the same output contract. The disagreement between families is exactly what you’re paying for.

What it looks like as a playbook

The whole thing is one playbook: a trigger on merge request events, seven isolated review stages that fan out in parallel, an aggregation stage where a model judges the reviews against your merge criteria, and a gate.

sumn runs watch review-panel/mr-2214

Two kinds of gate, and the difference is the whole design. The aggregation stage is a judged gate: a model applies criteria you wrote. The merge itself stays behind a held gate: a human clicks it, every time, until you decide otherwise. When the panel splits the way it did here, the run parks itself in your attention inbox with the split laid out, and the receipt shows what every stage read, wrote and cost.

The full seven-model panel on this diff cost less than two dollars, itemised per call. We know because the receipt is the first thing you see when the run finishes.