A code and security review that has to survive an argument.
Three language models from three different vendors review the same diff without seeing each other's findings. Then every finding only one of them raised goes back to the other two, who have to either agree or say specifically why the problem cannot occur. Whatever is still contested after that is arbitrated by a human reading the actual code. Only findings that survive reach the report.
This is a Claude Code skill: a single
SKILL.md that Claude follows, plus the shared brief the panelists are given.
Ask one model to review a diff and you get a list. Some of it is real. Some of it is a fluent, confident sentence about a vulnerability that cannot happen in this codebase, written in exactly the same tone as the findings that matter.
Asking the same model again does not help, because it agrees with itself. Telling it to be more critical does not help either: it becomes more critical about everything, including the parts that were fine. The problem is not that the model is bad at reviewing. It is that a single reviewer has nothing to disagree with.
So make it a panel, and make the panel argue.
| Stage | What happens |
|---|---|
| Round 1 | Deepseek, Kimi and Codex review the same diff concurrently. No panelist sees another's output. |
| Merge | Findings are clustered by file and line proximity. Two or more panelists independently = consensus. Exactly one = disputed. The clustering is mechanical, so a human corrects it: two descriptions of one root cause in different files will not cluster, and one line with two separate faults will cluster when it should not. |
| Round 2 | Each disputed finding goes to the panelists who did not raise it. They must agree, or refute and name why the issue cannot occur. Being contrarian is treated as exactly as bad as being credulous. |
| Termination | A rebuttal may raise something genuinely new, which goes back into the pool, but the debate is capped at two rounds. Everything unresolved is escalated rather than argued indefinitely. |
| Arbitration | A human opens the repository and rules on what is left: confirmed, refuted with the line that disproves it, or unverifiable. Majorities are not verdicts. Two models agreeing on a hallucination is still a hallucination. |
| Report | One verdict table, an exploit scenario and fix for every confirmed critical, a debate log, and an appendix of everything that died with its cause of death. Gate: fail on any confirmed critical. |
The rejected findings are the part that tells you whether to trust the accepted ones, which is why they are printed rather than dropped.
review-prompt.txt is given to all three panelists and is doing more work than
the choice of models. It fixes the output as a strict JSON schema so findings
can be compared mechanically rather than read as prose. It requires a category
and a written exploit scenario for every finding, which is the cheapest filter
available: a model that cannot describe how something is exploited usually
cannot, on inspection, explain why it is a finding at all. It carries an explicit
exclusion list for the false positives these tools raise every single time. And
it forces a calibrated confidence score, which the rebuttal round depends on.
Copy the directory into your Claude Code skills folder:
git clone https://github.com/IDSTUK/panel-review.git ~/.claude/skills/panel-reviewThen run /panel-review in Claude Code, optionally with a focus:
/panel-review auth.
Each seat is independent and optional. The skill probes for what is available, runs with whoever turns up, and names the empty seats in the report, because a three-model verdict and a one-model verdict should not look the same on the page.
| Seat | Needs |
|---|---|
| Deepseek V4 Pro | DEEPSEEK_API_KEY in the environment or .claude/settings.local.json |
| Kimi K3 | KIMI_API_KEY in the environment |
| OpenAI Codex | npm i -g @openai/codex and codex login |
The skill also falls back to reading a Kimi key from ~/.claude/skills/kimi-review/config.json
if that separate skill happens to be installed. It is not required; the
environment variable is enough.
No key is ever written to disk by this skill, and no key appears in the report.
- It reviews a diff, not a system. An architectural flaw spanning files the change never touches is invisible to it, and no amount of arguing will surface it.
- Three vendors is not statistical independence. These models share a great deal of training data and can be wrong in the same direction. Disagreement is evidence. Agreement is weaker evidence than it looks.
- Two of the three seats never see the repository. They reason from the diff alone, which is exactly why arbitration reads the real file before anything is called confirmed.
- It does not replace a review by someone accountable. It replaces the first pass, and it makes the human pass start from a much shorter and better argued list.
- It is not a compliance artefact. It is a working tool.
Written up, with the reasoning behind it, at idst.uk/security-audit.
MIT. See LICENSE.