Skip to content

v0.20260817.0

Latest
Latest

Choose a tag to compare

View all tags
@github-actions github-actions released this 17 Aug 19:24
· 18 commits to develop since this release
v0.20260817.0
2f9fbd4

NEW! Auto routing

TLDR: model="auto" picks the right model per request

Register a developer-authored routing map once at startup, then call any completion API with model="auto" (or model="auto/<label>" to pin a subtree). A local embedding classifier - the same multilingual ONNX model the semantic cache uses - reads a deterministic slice of the conversation, scores it against your example phrases, and resolves to a concrete provider/model plus fallback chain.

No API calls. No conversation data leaves the process.

import onellm
from onellm import ChatCompletion

onellm.init_routing({
   "default": "openai/gpt-5-mini",
   "code": {
       "examples": ["fix this function", "why does this test fail?"],
       "models": ["anthropic/claude-sonnet-4-5", "openai/gpt-5"],
   },
   "research": {
       "examples": ["compare these papers", "summarize this report in depth"],
       "models": "openai/gpt-5",
   },
})

response = ChatCompletion.create(model="auto", messages=[...])
response.routing  # {"resolved_path": "code", "model": "...", "scores": {...}, ...}

Highlights

  • You own the map. OneLLM ships no opinions about which model is good at what - labels, examples, and model choices are all yours.
  • Strictly additive. Nothing changes unless you call init_routing(). Concrete provider/model strings behave exactly as before.
  • Never fails to route. Every group requires a default; low-confidence requests fall back to it deterministically instead of erroring.
  • Fail-fast config. Map schema, providers, and credentials are validated at startup - not at 3am when a route first fires. Supply keys inline with api_keys={"openai": "env:OPENAI_API_KEY"}.
  • Fast. Classification costs a few ms; a memo LRU makes repeated turns on an unchanged conversation ~µs. acreate() runs inference in a thread executor so the event loop never blocks.
  • Observable. response.routing carries the full decision record; onellm.explain_route() dry-runs a decision with no provider call (assert it in CI); onellm.routing_stats() aggregates counters; ONELLM_ROUTING_DEBUG=1 logs what the classifier saw.

Install

pip install "onellm[routing]"

Aliases onellm[cache] - if you already use the semantic cache, routing adds no new dependencies and shares the loaded model. The base install is unchanged.

Dependency pins (CVE safety)

• cryptography>=50.0.0
• httplib2>=0.32.0
• pyasn1>=0.6.4
• h2>=4.4.1
• setuptools>=83.0.0 (build)

Docs

Full guide - map schema, YAML loading, credentials, tuning, and feature interactions: docs/routing.md

Full Changelog: v0.20260708.1...v0.20260817.0

Assets 2
Loading