Repository navigation
v0.10.0
llm-translator v0.10.0 Changelog
Release Date: November 2026
Release Type: Minor
Branch: next → master
Overview
Version 0.10.0 adds four things. Anthropic requests to vLLM can go to its own /v1/messages instead of being converted, a pool skips vLLM upstreams that aren't ready and routes to LoRA adapters, five hosted OpenAI-compatible providers (Groq, Together, Fireworks, xAI and Gemini) get drivers, and Ollama can be used through its native /api/chat protocol.
Everything is opt-in except the provider drivers, which bind by host name, and the readiness probes, which only run when a model has more than one endpoint. One behaviour change applies to upstreams with backend: "vllm" set by hand; see the upgrade notes.
None of this has been run against a real server yet. It's built from the providers' API docs and source and tested against fixtures.
✨ Features
Native /v1/messages for vLLM (#152)
vLLM 0.11.1 and later serve the Anthropic Messages API themselves. A new nativeMessages setting (true, false or "auto", default false) sends an Anthropic-front request to that endpoint with only normalisation, instead of converting it to Chat Completions and back. "auto" uses it when the detected or configured version is 0.11.1 or later, and converts when the version is unknown.
- The system prompt is sent as one string with the billing header block removed.
- vLLM sends no
pingevent, so one followsmessage_start. - Thinking blocks vLLM sends without a signature (before 0.17) get the translator's marker signature.
- With
toolErrorson, failed tool results carry the marker text, as in a conversion, since vLLM ignoresis_error. - The upstream is called with the bearer key only; the client's
x-api-keyandanthropic-betaheaders aren't forwarded.
A request ending in an assistant prefill, one with document blocks, and one with deferred tools (unless toolSearch is native) are still converted. An upstream that answers 404 to /v1/messages with something other than an Anthropic error is converted from then on, until the next detect(), and a fallback event is emitted once. status() shows whether an upstream is on the native endpoint. See docs/drivers.md. Resolves #31.
Skip unhealthy upstreams and tag LoRA adapters (#153)
When more than one endpoint serves a model, upstreams whose driver names a health path (vLLM: GET /health) are checked before the pool is ordered, and the ones that aren't ready go last. A lone endpoint is still tried.
- New
pool.healthInterval(milliseconds a verdict is trusted, default 10000,0turns probing off) andpool.healthTimeout(default 2000). endpoint.status()andstatus()carryready,readyReasonandcheckedAt. Ahealthevent with kindsdownandupfires when an upstream changes state, andhealth.check()probes every upstream right away.- vLLM lists each LoRA adapter in
/v1/modelsas a model whoseparentis the base model. Adapters are kept apart in the detection facts (facts.adapters), are routable like any other model, and carryparentinmodel.list()andGET /v1/modelson both fronts. An upstream with one base model and some adapters is still the single model unknown names fall back to. - A model list fetched live now updates what a vLLM upstream serves, so adapters loaded or unloaded at runtime start or stop being routable. See the upgrade notes.
See docs/drivers.md and docs/server.md. Resolves #32.
Presets for hosted OpenAI-compatible providers (#154)
groq, together, fireworks, xai and gemini drivers, bound by host name (api.groq.com, api.together.ai, api.fireworks.ai, api.x.ai, generativelanguage.googleapis.com) or chosen with backend for a proxy in front of the provider. The provider's path root is added for you, so the host alone works as the upstream url.
The provider facts sit in one table, PRESETS in src/lib/presets.mjs: the path root, the token field (max_tokens or max_completion_tokens), the reasoning field, how thinking and effort are sent, whether top_k is accepted, whether the stream takes stream_options, and the request fields the provider rejects (Groq's logprobs and logit_bias, xAI's stop and penalties on reasoning models). The driver folders are one-line wrappers over it, and settings on an upstream still overrides any value. See docs/drivers.md. Resolves #44.
Ollama's native /api/chat protocol (#155)
An api: "native" setting on an ollama upstream (default "openai", the existing /v1 path) talks to Ollama's own chat API, which exposes the controls /v1 hides: think, options and format. Both fronts are converted to and from it, streamed or not.
- Tool arguments are JSON objects and tool results are matched by
tool_name. Ollama sends no tool-call ids, so each call gets a generated one. thinktakes the effort level when there is one, else the thinking switch.message.thinkingis read back as a thinking block (or the reasoning field on the OpenAI front) and sent on assistant turns.- Sampling fields go into
options,response_formatandoutput_config.formatbecomeformat, and images go as base64 strings. - The stream is newline-delimited JSON, buffered across chunks so a line split at any byte is parsed once complete.
tool_choiceother thanauto,nabove 1,logit_biasandlogprobsare dropped and recorded instate.dropped. The model list and embeddings stay on/v1.
Setting api: "native" on an upstream bound to a driver without a native API is an error at detection. See docs/drivers.md and docs/protocol-support.md. Resolves #35.
📚 Documentation
- NEW: docs/changelog/v0/v0.10.0.md: this changelog.
- docs/drivers.md has new sections for the hosted providers, native
/v1/messagesfor vLLM, readiness and LoRA adapters, and Ollama's native mode. docs/settings.md, docs/server.md, docs/events.md and docs/protocol-support.md cover the new settings, pool options,healthevent and Ollama converters. docs/known-limitations.md and the feature inventory (FI-137, FI-139, FI-146) are updated to match.
🔧 Dependencies
No dependency updates.
Upgrade notes
- An upstream with
backend: "vllm"and nomodelslist now serves what the server last listed. An explicitbackendskips detection, so such an upstream had no model list of its own and a model list fetched from the server was never kept. Now every successful fetch (model.list(),GET /v1/models) replaces the upstream's served models, adapters included, with what the server returned, and requests for those models route to it. An upstream with a configuredmodelslist keeps that list, andbackend: "auto"upstreams get the same refresh on top of what detection found. - Pools probe vLLM upstreams. A model with more than one endpoint now gets a
GET /healthon each vLLM endpoint, at most once perpool.healthInterval. A server that answers something other than 2xx or 5xx (a proxy without that path) is left alone until the nextdetect(). Setpool.healthInterval: 0to turn probing off. - Provider hosts change driver.
api.groq.com,api.together.ai(andapi.together.xyz,us-api.together.ai),api.fireworks.ai,api.x.aiandgenerativelanguage.googleapis.comupstreams are now bound togroq,together,fireworks,xaiorgemini, where they ran oncompatible. Setbackend: "compatible"on the upstream to keep the old behaviour. nativeMessagesand Ollama'sapi: "native"are off unless you set them.- No other changes are needed.
| Metric | Coverage |
|---|---|
| Statements | 100.0% |
| Branches | 100.0% |
| Functions | 100.0% |
| Lines | 100.0% |
Avg: 100.0% · 60e4777 · Node lts/*