v12.0.0 — Chuzom capability wave

v12.0.0 — Chuzom capability wave: execution ledger, quality/fallback split, capability-aware shadow routing, budget envelope, misroute audit + CLI (2026-08-02)

Version note: released as a major bump (v11.0.0 → v12.0.0) to mark the scale of the Chuzom capability wave, even though every routing-affecting behavior ships default-off or shadow-only.

Ports eight capabilities adapted from an internal reference implementation ("Chuzom") into llm-router. Everything here is additive; every new routing-affecting behavior ships default-off or shadow-only, so existing installs behave identically until explicitly opted in.

Added

• Execution ledger + session store (execution_ledger.py, session_store.py). Append-only SQLite ledger recording every route attempt (execution_events) via additive ALTER TABLE migrations onto the existing usage.db, with realized-savings gating (see below) and route/cost invariants. session_store.py adds a durable JSONL session-context log with cross-process advisory locking, TTL/size-triggered compaction, and privacy modes.

• Quality/fallback split (routing_quality.py, bounded_operational.py, quality_feedback.py). A schema-v2 route-quality ledger (routing_quality.py) with fail-open recording and a summarize() that never conflates verified/unverified or legacy-v1 rows into v2 metrics. bounded_operational.py adds a bounded-operational route predicate and pricing-derived budget, gated by LLM_ROUTER_BOUNDED_OPERATIONAL (default off). quality_feedback.py gains LoopHole ground-truth verdict ingestion (record_loophole_verdict/ingest_loophole_jsonl) feeding the existing heuristic quality store.

• Realized-savings measurement + dashboard split (dashboard_data.py::query_realized_savings, dashboard/server.py). Realization-gated savings accounting alongside the existing potential-savings columns: only attempts with realization_status == "verified_used" and an adoption method that counts as realized are counted, and the figure is never reconciled against or allowed to overwrite usage.saved_usd. Exposed additively as /api/stats's realized_savings key, isolated in its own fail-open block.

• Capability-aware routing (shadow mode) (capabilities.py, wired into router.py/cost.py). An 8-bit capability detector records what capability-aware routing would choose into routing_decisions.capabilities_json, without changing any live routing decision. Gated by LLM_ROUTER_CAPABILITY_ROUTING (default off); live routing (needs_claude_tools()) stays byte-identical regardless of the flag.

• Budget envelope (budget_envelope.py). Standalone BudgetEnvelopeManager (register/reserve/release/commit/settle/tier-state, hierarchical ancestor accounting) gated by LLM_ROUTER_BUDGET_ENVELOPE (default off). Ships as an accounting primitive only — no router/cost wiring — so routing and spend behavior are unchanged with the flag off; execution_ledger.py remains the sole source of truth for realized spend.

• Misroute audit (audit_routing.py). A fully offline, post-hoc scorer over existing routing_decisions rows (heuristic over judge score / complexity downgrades / downshifts), writing back new audit_verdict/audit_checked_at columns idempotently. Gated by LLM_ROUTER_AUDIT_DISABLED; inert until explicitly invoked via run_audit() or the new llm-router audit CLI command (below).

• Retrospective loop + team report enrichment. Verified the existing retrospective debrief (retrospective.py, native since v6.x) reads the new audit_verdict directly rather than re-deriving misroutes, and fails open on the context fields it reads from the items above. commands/team.py's report/push surfaces gain fleet-wide realized-savings and inferred misroute-rate columns, sourced from the realized-savings query and quality-ledger summary via a fail-open helper.

• llm-router audit CLI command (commands/audit.py). Wires audit_routing.py::run_audit() to a CLI entry point (mirrors the team command's structure): renders sampled/audited counts, verdict breakdown, and the inferred misroute-rate baseline, with a --json mode and a --limit N flag (default 100). Respects LLM_ROUTER_AUDIT_DISABLED. Strictly read-only/reporting — never mutates routing state.

• Bounded-operational routing wired into the live path (router.py), strictly behind LLM_ROUTER_BOUNDED_OPERATIONAL (default off). When the flag is unset/false, the routing decision path is byte-identical to before, proved by an invariance test comparing route decisions with the module absent vs. present-but-disabled.

Config

New env vars (all optional, all default off / non-disabling): LLM_ROUTER_BOUNDED_OPERATIONAL, LLM_ROUTER_LOOPHOLE_JSONL, LLM_ROUTER_CAPABILITY_ROUTING, LLM_ROUTER_BUDGET_ENVELOPE, LLM_ROUTER_AUDIT_DISABLED (opt-out — unset means audits run when explicitly invoked).

Notes

• There is no LLM_ROUTER_QUALITY_FEEDBACK flag, and none is planned. The heuristic quality scorer's should_skip_model() check in the router's fallback-chain path is unconditional — it pre-dates this release, is unrelated to the LoopHole-verdict additions above, and is already always-on in production. Gating it now would change existing behavior, so it intentionally stays ungated; this note exists only to correct an earlier reference to a flag that was never implemented.