Repository navigation
v0.11.0
llm-translator v0.11.0 Changelog
Release Date: November 2026
Release Type: Minor
Branch: next → master
Overview
Version 0.11.0 adds three things and fixes one. context_management clear_tool_uses edits are now emulated for OpenAI-protocol upstreams, a neutral constraint (JSON Schema, regex, choice or grammar) is turned into each backend's native constrained-decoding field, and the vLLM cache_salt and OpenAI prompt_cache_key hints are mapped. The fix makes upstreams on a known hosted API bind by host name before any driver probes them.
Everything new is opt-in except the host-first binding and the clear_tool_uses edit, which only applies when a client sends one. The cache and constraint field names come from each backend's documentation and have not been run against a real server.
🐛 Bug Fixes
Host-matched upstreams bind before any driver probes (#163)
A driver can name the hosts it owns in info().hosts, or a host suffix in info().hostSuffixes. Detection now tries those drivers first, whatever their order, and binds the upstream with no request at all. Before, a hosted API such as api.openai.com was probed as vLLM (GET /version, GET /v1/models) and other self-hosted servers before its own driver got a turn. The openai, deepseek, mistral and openrouter drivers declare their hosts, azure declares its three resource suffixes, and the presets for Groq, Together, Fireworks, xAI and Gemini do the same. Every other upstream is probed in order as before. See docs/drivers.md. Fixes #158.
✨ Features
clear_tool_uses context edits (#162)
An Anthropic request carrying a clear_tool_uses_20250919 edit in context_management has it applied locally, on the converted messages, before it goes to an OpenAI-protocol backend. The reply reports what was cleared in context_management.applied_edits, streamed or not.
trigger(input_tokensortool_uses, default 100000 input tokens),keep(default the 3 most recent tool uses),clear_at_least,exclude_toolsandclear_tool_inputsare honoured.- Clearing goes from the oldest tool use forward. A cleared result becomes
[Tool result cleared to save context], the call stays, and withclear_tool_inputs: trueits arguments become{}. - Input tokens are estimated at one per four characters, so a trigger can fire somewhat earlier or later than on Anthropic.
- An edit with parameters outside the documented shape is dropped as a whole. Compaction and unknown edit types are still dropped and recorded in
state.dropped.
clear_thinking was already emulated. See docs/protocol-support.md and docs/known-limitations.md. Resolves #13.
Native constrained decoding (#164)
A neutral constraint describes one of json (a JSON Schema), regex, choice (a list of strings) or grammar. Set it per upstream with the constraint setting, or per request as constraint on the OpenAI front and metadata.constraint on the Anthropic front. The driver of the upstream turns it into the backend's own field, and the extension is never forwarded under its own name.
| Backend | json |
regex |
choice |
grammar |
|---|---|---|---|---|
vLLM 0.11.0 and later, unknown version, compatible |
structured_outputs |
structured_outputs |
structured_outputs |
structured_outputs |
| vLLM before 0.11.0 | guided_json |
guided_regex |
guided_choice |
guided_grammar |
| SGLang | response_format |
regex |
regex |
ebnf |
| llama.cpp | json_schema |
not supported | grammar |
grammar |
| TGI | response_format |
response_format |
response_format |
not supported |
| other OpenAI-protocol | response_format |
not supported | not supported | not supported |
A kind the backend cannot enforce follows unknownFields: dropped and recorded as constraint, or rejected. vLLM applies a structured-outputs constraint to the whole generation, so when a request offers tools the translator sends the tools and drops the constraint. A request sent to vLLM's own /v1/messages (nativeMessages) does not get the constraint either. The OpenAI front's passthroughFilter now lets vLLM's structured_outputs and guided_* fields, and llama.cpp's grammar and json_schema, through for those backends only. See docs/drivers.md. Resolves #14.
Prompt-cache hints (#165)
Two hints are mapped, and each goes to one backend only. Everything else about caching is unchanged: cache_control is still dropped and cache_creation_input_tokens is still always 0.
cache_salt(vLLM only). vLLM mixes the salt into its prefix-cache hash, so requests with different salts never share cached prefixes. It is set by the server, never by a client: thecacheSaltsetting per upstream, or thecacheSaltcall option ofrequestandsend, which wins. For a per-API-key salt, anauthorizehook returns{ ok: true, cacheSalt }. Acache_saltin a client body is never used. Other drivers drop it and recordcache_salt, or reject it underunknownFields: "error".prompt_cache_key(OpenAI only). Sent to theopenaidriver from the client's own field,metadata.prompt_cache_keyon the Anthropic front, or thepromptCacheKeysetting for requests with none. Other drivers drop it and record it.prompt_cache_retentionis not mapped.
docs/server.md has a new Prompt caching section on what is and isn't cached, with guidance on keeping prefixes stable. See docs/server.md. Resolves #15.
📚 Documentation
- NEW: docs/changelog/v0/v0.11.0.md: this changelog.
- docs/drivers.md has a Constrained decoding section and the host-first detection order. docs/server.md has Prompt caching, and docs/settings.md lists
constraint,cacheSaltandpromptCacheKey. docs/protocol-support.md, docs/known-limitations.md and the feature inventory (FI-031, FI-035, FI-040, FI-100, FI-136, FI-138) are updated to match.
🔧 Dependencies
prettier3.9.10 → 3.10.1 (#159), a dev dependency.
Upgrade notes
- Upstreams on known hosts bind without probing.
api.openai.com, the DeepSeek, Mistral and OpenRouter hosts, Azure OpenAI resources and the hosted-provider presets now bind to their own driver at creation with no request. Setbackendon the upstream to choose another driver. cache_saltis server-side only. A client cannot set it; use thecacheSaltsetting, the call option or theauthorizehook.- No other changes are needed.
| Metric | Coverage |
|---|---|
| Statements | 100.0% |
| Branches | 100.0% |
| Functions | 100.0% |
| Lines | 100.0% |
Avg: 100.0% · d730c97 · Node lts/*