Skip to content

v0.8.0 — Layer-2 semantic routing, learning loop, dashboard + -semantic image

Choose a tag to compare

@izzoa izzoa released this 22 Jul 01:41

Highlights

  • Layer-2 semantic routing (opt-in, flag-gated). A local ONNX embedding classifier resolves what the structural layer (L1) finds ambiguous: the request is embedded (~5–20 ms, CPU) and scored against bundled anchor centroids, and a confident high/low verdict routes via the existing auto_high/auto_low targets. Anything ambiguous or faulted degrades to cascade/default unchanged — the smart path never fails a request.
  • Per-tenant learning loop. Learned centroids track each tenant's own cascade outcomes (opt-in, default OFF). Evidence accumulates only as ≥ cohort-size aggregates in Redis — never a single raw embedding — and a daily sweep folds it under strict rails (capped EMA, spherical drift clamp toward the bundled anchors, cooldown, revision match), crash-atomically across Redis + Postgres. Learned supersedes bundled only behind read-time gates; any fault falls back to bundled. One-click Revert to bundled.
  • Dashboard surfaces. The old permanently-locked "L2 · Semantic" stub is now a real driven toggle with honest available/unavailable copy; a learning card shows the toggle, fresh-sample/source/last-applied status, the numeric audit history, and a confirmed revert; Auto-performance gains the semantic slice (evaluated, routed-per-band, four-way outcome split, bundled/learned source) with residual-cascade labeling; the request inspector carries a semantic_source chip.
  • Batteries-included -semantic image. ghcr.io/izzoa/polyrouter:0.8.0-semantic ships the ONNX runtime and the reference all-MiniLM-L6-v2 model (Apache-2.0) pre-baked with SEMANTIC_MODEL_PATH preset — L2 is available on first boot, nothing fetched at runtime. The baseline image stays ORT- and model-free.
  • max_tokens fix. Local / legacy OpenAI-compatible endpoints that accept only max_tokens no longer lose the caller's output-token cap.

Upgrade notes

  • Migrations run on boot (0020 semantic telemetry columns; 0022 the semantic_learning_event table). Back up the database first, as always.
  • Semantic routing is off by default; the baseline image is unchanged. To enable it, either run the 0.8.0-semantic image (ORT + model pre-baked) or install onnxruntime-node@1.27.0 and set SEMANTIC_MODEL_PATH to a model bundle. Add semantic to ROUTING_AUTO_LAYERS to expose the capability; tenants opt in per-account. New SEMANTIC_* / SEMANTIC_LEARNING_* knobs are all defaulted (learning stays off until a tenant enables it). No prompt text or embedding vector is ever logged or persisted.
  • local providers now emit max_tokens (not max_completion_tokens). Override per provider via the new max-tokens field if your gateway expects the other spelling.

Full package changelogs live in packages/*/CHANGELOG.md.