v0.8.0 — Layer-2 semantic routing, learning loop, dashboard + -semantic image
Highlights
- Layer-2 semantic routing (opt-in, flag-gated). A local ONNX embedding classifier resolves what the structural layer (L1) finds ambiguous: the request is embedded (~5–20 ms, CPU) and scored against bundled anchor centroids, and a confident
high/lowverdict routes via the existingauto_high/auto_lowtargets. Anything ambiguous or faulted degrades to cascade/default unchanged — the smart path never fails a request. - Per-tenant learning loop. Learned centroids track each tenant's own cascade outcomes (opt-in, default OFF). Evidence accumulates only as ≥ cohort-size aggregates in Redis — never a single raw embedding — and a daily sweep folds it under strict rails (capped EMA, spherical drift clamp toward the bundled anchors, cooldown, revision match), crash-atomically across Redis + Postgres. Learned supersedes bundled only behind read-time gates; any fault falls back to bundled. One-click Revert to bundled.
- Dashboard surfaces. The old permanently-locked "L2 · Semantic" stub is now a real driven toggle with honest available/unavailable copy; a learning card shows the toggle, fresh-sample/source/last-applied status, the numeric audit history, and a confirmed revert; Auto-performance gains the semantic slice (evaluated, routed-per-band, four-way outcome split, bundled/learned source) with residual-cascade labeling; the request inspector carries a
semantic_sourcechip. - Batteries-included
-semanticimage.ghcr.io/izzoa/polyrouter:0.8.0-semanticships the ONNX runtime and the referenceall-MiniLM-L6-v2model (Apache-2.0) pre-baked withSEMANTIC_MODEL_PATHpreset — L2 is available on first boot, nothing fetched at runtime. The baseline image stays ORT- and model-free. max_tokensfix. Local / legacy OpenAI-compatible endpoints that accept onlymax_tokensno longer lose the caller's output-token cap.
Upgrade notes
- Migrations run on boot (
0020semantic telemetry columns;0022thesemantic_learning_eventtable). Back up the database first, as always. - Semantic routing is off by default; the baseline image is unchanged. To enable it, either run the
0.8.0-semanticimage (ORT + model pre-baked) or installonnxruntime-node@1.27.0and setSEMANTIC_MODEL_PATHto a model bundle. AddsemantictoROUTING_AUTO_LAYERSto expose the capability; tenants opt in per-account. NewSEMANTIC_*/SEMANTIC_LEARNING_*knobs are all defaulted (learning stays off until a tenant enables it). No prompt text or embedding vector is ever logged or persisted. localproviders now emitmax_tokens(notmax_completion_tokens). Override per provider via the new max-tokens field if your gateway expects the other spelling.
Full package changelogs live in packages/*/CHANGELOG.md.