Repository navigation
v0.9.0
llm-translator v0.9.0 Changelog
Release Date: November 2026
Release Type: Minor
Branch: next → master
Overview
Version 0.9.0 adds four drivers: Azure OpenAI, OpenRouter, Hugging Face Text Generation Inference (TGI) and LocalAI. All four used to run on the generic compatible driver at best. Azure OpenAI didn't work at all without rewriting URLs by hand. Each driver now handles its server's own URL layout, auth, reply quirks and error shapes.
To make Azure possible, a driver can now build its own request path through an optional path leaf, and api-key joins bearer and x-api-key as an auth style. Upstreams on these servers move to their new drivers automatically, and self-hosted servers get a few more startup probes. See the upgrade notes.
None of the four has been run against a real server yet. They're built from the providers' API docs and source and tested against fixtures.
✨ Features
Azure OpenAI driver (#145)
An azure driver, matched by host name (*.openai.azure.com, *.cognitiveservices.azure.com, *.services.ai.azure.com) with no request of its own. The key goes in the api-key header.
- By default requests go to the v1 API (
<endpoint>/openai/v1/chat/completions, the model named in the body, noapi-version). - Set
apiVersionon the upstream to use the classic URLs instead:<endpoint>/openai/deployments/<deployment>/chat/completions?api-version=<version>.deploymentis a second new upstream field and defaults to the request's model. The model list and embeddings follow the same layout. - A base URL pasted with
/openaior/openai/v1on the end works too. - Token limits go out as
max_completion_tokens. With anapiVersiondated before 2024-09-01 they go out asmax_tokens, the only field those versions know. - A prompt rejected by the content filter (HTTP 400,
code: "content_filter") keeps that code. Its message lists the categories that were filtered, with their severity. Filtered replies already gotstop_details.categoryin v0.8.0.
Entra ID tokens aren't built in; send one through the upstream's headers. See docs/drivers.md. Fixes #43.
OpenRouter driver (#146)
An openrouter driver for openrouter.ai, matched by host name. https://openrouter.ai, https://openrouter.ai/api and https://openrouter.ai/api/v1 all work as the base URL.
reasoning_details(reasoning.text,reasoning.summaryandreasoning.encrypteditems) comes back to OpenRouter unmodified and in order on the next turn, streamed or not.- OpenRouter can answer HTTP 200 with an
erroron the choice instead of a message. That's now a real upstream error with the provider's status and error type, and in a stream it ends the stream with anerrorevent. - Reasoning effort is sent as
reasoning: { effort }, and Anthropic thinking with abudget_tokens(and no explicit effort) asreasoning: { max_tokens }. Areasoningobject an OpenAI client sends itself is passed through as is. - The final usage chunk and the
: OPENROUTER PROCESSINGkeep-alive comments are handled.
Fixes #41.
Hugging Face TGI driver (#147)
A tgi driver, detected by a strict match on GET /info. It needs router: "text-generation-router", a string model_id and version, and a numeric max_total_tokens, so Text Embeddings Inference, which answers /info as text-embeddings-router, isn't claimed. vLLM now steps aside for it.
TGI's replies are normalised for the deviations older releases are known for:
tool_callssent as a single object;- numeric tool-call ids, and empty ids on streamed fragments, which used to open a new
tool_useblock per fragment; - tool arguments as an object instead of a JSON string;
- the
eos_tokenandstop_sequencefinish reasons; - the
text_completionobject name.
Structured output is sent in TGI's grammar shape: a JSON schema becomes response_format: { type: "json_schema", value: { name, schema } }. A client's own grammar is accepted and folded into response_format. TGI refuses tools and a grammar together, so tools win and the grammar is dropped. Fixes #38.
LocalAI driver (#148)
A localai driver, detected by the <version> (<commit>) shape of LocalAI's GET /version, or by its /.well-known/localai.json discovery document. vLLM, which also answers /version, steps aside for it.
- Reasoning is read from LocalAI's
reasoningfield (andreasoning_content) and sent back asreasoning. - LocalAI sets parallel tool calls per model in its YAML (
function.parallel_calls) and ignoresparallel_tool_calls. When an Anthropic client setsdisable_parallel_tool_use, the driver keeps only the first tool call of each reply and stream. An OpenAI client'sparallel_tool_calls: falseis forwarded but has no effect on LocalAI. repetition_penaltyis sent asrepeat_penalty, and LocalAI's extra sampling fields (top_k,min_pand others) are passed through.
docs/drivers.md covers parallel_calls and the case where a model emits tool calls as plain text, with the LocalAI function: settings that fix it. Fixes #39.
📚 Documentation
- NEW: docs/changelog/v0/v0.9.0.md: this changelog.
- docs/drivers.md has new Azure OpenAI, OpenRouter, Hugging Face TGI and LocalAI sections and the updated detection table. The README's upstream config covers
apiVersionanddeployment. docs/known-limitations.md and the feature inventory (FI-149, FI-150, FI-152, FI-154) are updated to match.
🔧 Dependencies
No dependency updates. @cldmv/slothlet and @cldmv/slothlet-types stay at 3.23.0.
Upgrade notes
- Upstreams on these servers change driver. An Azure OpenAI or OpenRouter host is now bound to
azureoropenrouter, and a TGI or LocalAI server totgiorlocalai, where they used to run oncompatible(or, for TGI and LocalAI, could be taken for vLLM). Setbackend: "compatible"on the upstream to keep the old behaviour. - Azure upstreams that rewrote URLs by hand. If you pointed an upstream at a full deployment URL to make
compatiblework, point it at the resource endpoint instead and setapiVersion(anddeploymentif it differs from the model name). - More startup probes. A server no earlier driver claims now also gets
GET /infoandGET /.well-known/localai.json, and vLLM's own check also asks for/infowhen/versiondoesn't name vLLM. The probes run in parallel and are bounded bydetectTimeout. Setbackendon an upstream to skip probing. - No other changes are needed.
| Metric | Coverage |
|---|---|
| Statements | 100.0% |
| Branches | 100.0% |
| Functions | 100.0% |
| Lines | 100.0% |
Avg: 100.0% · 05d19bf · Node lts/*