Skip to content

v2.2.0 — Tiered routing: measured and declined

Choose a tag to compare

@sanlee-ys sanlee-ys released this 18 Jul 14:06
· 134 commits to main since this release
8975b83

Milestone: tiered model routing — measured and declined (ADR-013). The project's second measured negative result, shipped as such.

The shipped classifier forces tool use and emits no confidence signal, so the routing layer manufactures one: a routing-only tool variant requires a runner_up_category, and escalation to Opus fires exactly when the top-two are {technology, operations} — the one clustered confusion the gold set ever showed. Measured with three honesty guards (human-gold-only quality grading, zero-new-spend escalation replay, schema perturbation measured rather than assumed away):

  • Routing moved +0 rows on both axes (94.4% / 92.6%, identical to the workhorse); escalated rows read fixed 0, broke 1, unchanged 8 — the one change was Opus importing the judge's own known error.
  • Even all-Opus scores the same 94.4% category — no category router has headroom on this set.
  • ~1.97x cost per article at a 19.4% real-world escalation rate, for zero measured quality.

Routing is declined; the harness stays dormant as the reproducible record. Also fixed: bulk passes now record-and-continue on safety-layer refusals (the s151 finding — a schema change alone flipped a benign chem-bio defense story into a refusal).

Full detail: CHANGELOG 2.2.0