Turn one hard "which should I do?" into a judged tournament — and get back a defensible winner.
Terminal summary from a real run — verdict and scores are from the saved report, re-rendered at a readable pace.
bakeoff is a Claude Code skill. Hand it a decision and it generates diverse candidate solutions, auto-derives the criteria that matter for that specific problem (so you don't have to know what to score on), judges every candidate with independent scorers, and returns the winner plus a ranked shortlist — with the reason each one won or lost. You just type your request; it infers the shape (a comparison, an improvement, an idea, a "what if we do X", a problem, or a scoping call), the depth, and the rubric — no flags.
The hard part of any comparison isn't the scoring — it's knowing what to evaluate. bakeoff derives the rubric for you. Here's the run shown in the GIF above:
/bakeoff "8h/24h email deadline: Batches vs keep-sync vs merge?"
Roles → status-quo · cost-first · dedup · max-savings
Rubric → Deadline-fit 28 · COGS 24 · Personalization 16 ·
Complexity 16 · Ops 12 · Reversibility 4 [approve? yes]
Judges ×2 → reconcile → shortlist
B keep sync per-cohort 73 wins deadline-fit; 0 on COGS
A Message Batches 70 50% off, tiny in absolute $
C embedding merge 69
D hybrid 55
Refute → both judges picked A, but the 50% cut is half of an
already-tiny bill (~$30–150/mo) → flips to B
Winner → B: keep sync now; escalate 24h projects past ~$200/mo
You never supplied the six dimensions or their weights — and the adversarial pass caught that the judges' pick rested on a saving too small to matter. That's the point.
- Select, then graft. Diverse candidates + judge-based selection is the core. But when the top two are strong on different dimensions, it grafts the runner-up's best element into the winner — kept only if it re-scores above the best single candidate. In practice the graft wins or sharpens the pick about as often as it's discarded.
- Diversity is the biggest lever. Each candidate generator gets a distinct, problem-specific role (e.g.
cost-firstvsstatus-quo), so the field genuinely spans the space. - Independent judges, mechanically reconciled — not a debate (debate amplifies bias). Two judges score independently; a deterministic script merges them with a lower-score rule on disagreements.
- Position-bias controlled. Each judge sees the candidates in a different shuffled order, referenced by stable IDs.
- The leader gets stress-tested. Before committing, an adversarial pass actively tries to refute the top candidate. A plausible-but-wrong winner shouldn't survive.
- Grounded when it matters. For decisions that hinge on real facts (a library version, your actual codebase), it reads/searches before judging — and flags any part it couldn't verify as "training-knowledge only."
Pick one — all three install the same self-contained skill.
Agent Skills CLI (works with Claude Code, Codex, Cursor):
npx skills add CoriChui/bakeoffClaude Code plugin (marketplace install, auto-updates):
/plugin marketplace add CoriChui/bakeoff
/plugin install bakeoff@bakeoff
Manual — clone straight into your skills directory:
git clone https://github.com/CoriChui/bakeoff.git ~/.claude/skills/bakeoffThen in Claude Code:
/bakeoff "which caching strategy should we use for the API layer?"
That's it. The skill is self-contained — the rubric-builder, scorer, and reconciliation script are bundled (adapted from the evaluate skill). No other skills required.
- Claude Code (skills support).
- Node.js on your
PATH— the score reconciler (scripts/reconcile-scores.js) runs under Node.
Reach for bakeoff when all three hold:
- Wide solution space — several genuinely defensible approaches, not one obvious answer.
- Costly to reverse — a wrong call is expensive to unwind.
- Unclear criteria — you can't easily say why one option should beat another.
Good fits: architecture choices, library/database/tool selection, refactor strategies, migration approaches, "is the AI's suggestion actually good, or is there something better?" — and you needn't phrase any of them as a "decision." Any request shape qualifies: a comparison ("X vs Y"), an improvement ("best way to X"), an idea ("what should we build"), a proposal ("what if we do X"), a problem ("how do we handle this"), or a scoping call ("minimal X before launch"). bakeoff recognizes the shape itself.
Don't use it when a test, type-check, or lint settles the question, or when you're scoring a single artifact with no alternatives — that's a job for a plain evaluation, not a tournament. (Low-stakes or single-obvious-answer "which/how" questions are a direct answer, not a tournament — the three-part gate above still governs.)
You just type your request. bakeoff infers the rest:
- Entry point — it almost always generates the candidates itself. If your prompt names a concrete option or plan ("should the hero be GitHub-only?", "A vs B vs C"), that becomes a candidate and it invents rivals around it. Either way it always adds rivals — you never just judge what you brought.
- Floor candidate — for a proposal / idea / scoping request it always includes a genuine defer / do-the-minimal / status-quo candidate, and the rubric rewards restraint, so "not yet" can win (in real runs it often does).
- Depth — auto-scaled to the grounded stakes of the call, not the surface of the question:
| Depth | Candidates | Judges | Refute | Synthesis | Auto-picked when… |
|---|---|---|---|---|---|
| lean | 3 | 1 | no | no | narrow · low grounded magnitude · reversible |
| default | 4 | 2 + reconcile | yes | attempted | several defensible options · material stakes · recoverable |
| thorough | 5–6 | 2 + reconcile | yes | yes | irreversible · prod · data-model / public-API / migration / security |
Power users can still override — --lean / --thorough (depth), --seed <plan> (force a plan as Candidate A), --compare (judge pasted candidates) — but none are required. See SKILL.md for the full pipeline.
FRAME → { GENERATE K candidates ∥ BUILD RUBRIC } → [rubric gate] →
JUDGE (2 scorers ∥, randomized order, reconciled) →
RANK → ADVERSARIAL CHECK (only if the top-two are close or the leader is suspect) →
winner + top-N shortlist (+ optional synthesis) → REPORT
The rubric is built in parallel with candidate generation and blind to the candidates — so it describes the decision, not whichever option it might otherwise favor. Every run ends with a saved report (docs/bakeoffs/YYYY-MM-DD-<slug>.md) containing the recommendation, the shortlist, the full score matrix, judge agreement, and the rubric — so the decision is auditable and reusable later.
See full saved reports — a real run and an illustrative one — in examples/.