Skip to content

v11.0.0 — adaptive routing wave

Latest
Latest

Choose a tag to compare

View all tags
@ypollak2 ypollak2 released this 09 Jul 16:06
v11.0.0
181e8d1
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Ports a wave of routing + observability capabilities into llm-router. Everything new is additive and off-by-default where it touches routing, so existing setups behave identically until opted in.

Added

  • llm_router.observability.surface_status — cross-surface "router is working" status line / terminal title / OS notification (stdlib-only, fail-soft) for hosts without a native statusline.
  • llm_router.observability.summary — session-end summary over usage.db: savings vs baseline, tier mix, per-provider cost, latency p50/p95/p99, top routes. render_markdown() + optional rich render().
  • llm_router.classify — the deterministic signal scorer extracted into an importable module (classify_signals), so the router/gateway/tools can classify at 0 cost/latency. Byte-identical to the hook, drift-guarded.
  • RoutingProfile.SUBSCRIPTION_LOCAL — cost-inverted routing for "one paid seat + free local bucket": free-first for simple/moderate, seat-first for complex, seat demoted when its quota is strained. No-op unless LLM_ROUTER_SUBSCRIPTION_PROVIDER is set.
  • llm_router.signals — PiiSignal + force_local_for_pii: a prompt containing a secret routes to local models only, fail-closed so it never reaches an external API.

Changed

  • Docs restructure. docs/ is now gitignored (local notes) except the CI-generated docs/BENCHMARKS.md. README media → assets/readme/, public pages → guide/. External links to docs/*.md should move to guide/*.md.

Config

New optional env vars: LLM_ROUTER_SUBSCRIPTION_PROVIDER, LLM_ROUTER_INTERNAL_PROVIDERS, LLM_ROUTER_SUBSCRIPTION_PRESSURE_THRESHOLD, LLM_ROUTER_SUBSCRIPTION_REORDER_ALL_PROFILES, LLM_ROUTER_STATE_DIR, LLM_ROUTER_INDICATOR.

Tests & audit

97 new tests; full suite green (0 failures/errors). A pre-release audit caught and fixed two real bugs: an empty base chain for the explicit SUBSCRIPTION_LOCAL/QUOTA_BALANCED profile, and an ambiguous reorder tier when the seat provider was also in the free bucket.

Full details: CHANGELOG.md · pip install -U llm-routing

Assets 2
Loading