Skip to content

Releases

v0.13.0

Choose a tag to compare

@cldmv-bot cldmv-bot released this 19 Nov 16:27
v0.13.0
8da6da9

llm-translator v0.13.0 Changelog

Release Date: November 2026
Release Type: Minor
Branch: next → master


Overview

Version 0.13.0 adds the OpenAI Responses API on both sides of the translator. Clients such as Codex CLI can call /v1/responses, and an upstream can be reached over its own Responses API.

The upstream protocol is now one per-upstream api setting, and its default changes from Chat Completions to auto. Under auto each request goes over the client's own protocol when the server speaks it, and over Chat Completions otherwise. The most visible effect is that Claude Code pointed at vLLM 0.11.1 or later now uses vLLM's /v1/messages with no setting. Set api: "chat" on an upstream to keep the old behavior.


✨ Features

Responses API front (#174)

The OpenAI front serves POST /v1/responses and POST /v1/responses/input_tokens. Requests are stateless: input items, instructions, function tools, reasoning and structured output are converted, and the reply or SSE stream comes back as a Responses object or semantic events (response.created, response.output_text.delta, response.completed, and so on). Stateful and hosted features (previous_response_id, store: true, hosted tools) are refused with a 400. See docs/protocol-support.md. Resolves #6.

Responses API as an upstream (#173)

An upstream can be reached over /v1/responses from either client protocol. Requests are always sent with store: false, and encrypted reasoning items are carried between turns so reasoning models keep their context. The new reasoningSummary setting asks the server for reasoning summaries. A server that answers 404, 405 or 501 to /v1/responses drops back to Chat Completions, which is remembered until the next detection and reported as a fallback event. See docs/protocol-support.md. Resolves #7.

One api setting for the upstream protocol, auto by default (#177)

api takes a protocol id or an alias:

Protocol Value Aliases
Anthropic Messages anthropic.messages messages, anthropic, claude
OpenAI Chat Completions openai.chat chat, openai, chatgpt
OpenAI Responses openai.responses responses, codex
Ollama native /api/chat ollama.chat native, ollama
Detect auto (default)

Each driver lists the protocols its server speaks; vLLM, for example, speaks Messages, Chat Completions and Responses. Under auto a request uses the client's own protocol when the driver lists it, then Chat Completions. A 404, 405 or 501 moves that upstream to the next protocol until the next detection. vLLM's /v1/messages is used when the detected version is 0.11.1 or later, with the existing vLLM fixes and the conversion of bodies that endpoint can't take. An explicit value never falls back, and a value the driver doesn't list fails detection with unsupported_api. status() reports the protocol in use and the upstream's protocol list.

nativeMessages still works as a deprecated alias (true → anthropic.messages, "auto" → auto, false → openai.chat) and logs a deprecated detect event. api wins when both are set. See docs/drivers.md. Resolves #175.

🐛 Bug Fixes

  • reasoningRoundTrip after a fallback (#177): it was turned on for every request to a Responses upstream and stayed on after the upstream dropped to Chat Completions. It now follows the protocol each request actually uses, and an explicit setting still wins.
  • Token counting on vLLM (#177): a token count for an Anthropic client could pick the native /v1/messages route and send the Anthropic body to /tokenize. Token counts now always use the Chat Completions body.

📚 Documentation


Upgrade notes

  • The default upstream protocol is now auto. A Chat Completions client still gets Chat Completions on every driver. An Anthropic client now reaches vLLM 0.11.1 and later over /v1/messages, and a Responses client reaches a server that has /v1/responses over it. To keep Chat Completions for every client, set api: "chat" on the upstream.
  • api: "openai" and api: "native" keep their meaning as aliases of openai.chat and ollama.chat. An existing api: "auto" now prefers the client's protocol rather than always trying the Responses API first.
  • nativeMessages is deprecated. Replace nativeMessages: true with api: "messages" and nativeMessages: false with api: "chat", or remove it to get auto.
  • status() reports api and protocols per upstream. protocol, responses and messages are still there.

coverage

Metric Coverage
Statements 100.0%
Branches 100.0%
Functions 100.0%
Lines 100.0%

Avg: 100.0% · cb8905d · Node lts/*

👥 Contributors

v0.12.0

Choose a tag to compare

@cldmv-bot cldmv-bot released this 18 Nov 02:54
71f196e

llm-translator v0.12.0 Changelog

Release Date: November 2026
Release Type: Minor
Branch: next → master


Overview

Version 0.12.0 is about running the server in production. It adds an optional Prometheus /metrics endpoint, and TLS, per-key rate limits and CORS for the built-in server. All four are off unless you turn them on, so an existing deployment behaves as before.

One default changes: a failed tool result sent to a Chat Completions upstream is now marked with toolErrorMarker out of the box. Set toolErrors: false to keep the old behavior.


✨ Features

Prometheus metrics (#168)

GET /metrics serves Prometheus text format (0.0.4) with no extra dependency. Turn it on with --metrics, LLM_TRANSLATOR_METRICS=1, "server": { "metrics": true } in the config file, or the metrics: true option of server.start and server.plugin. When it's off, /metrics is an unknown route. The endpoint uses the same apiKey / authorize authentication as every other route.

Metric Type Labels
llm_translator_requests_total counter front, route, driver, code
llm_translator_request_duration_seconds histogram front, route, driver, code
llm_translator_tokens_total counter driver, type
llm_translator_errors_total counter kind
llm_translator_upstream_up gauge upstream, driver

Labels are bounded: route patterns, never URLs, and never models, request ids or keys. A client that disconnects is counted as code="499" and not as an error. Counters are in memory per server. Requests to unmatched paths and the scrapes themselves are not counted. See docs/server.md. Resolves #51.

TLS, per-key rate limits and CORS (#170)

Each is a server.start option and a server key in the config file, and each has CLI flags and environment variables. cors and rateLimit also work on the Fastify plugin.

  • TLS: tls: { key, cert, ca, passphrase } takes file paths and serves HTTPS. Flags are --tls-key, --tls-cert, --tls-ca and --tls-passphrase. A missing or unreadable file, or key without cert, fails start-up with invalid_config. Files are read once, with no reload.
  • Rate limit: rateLimit: { requests, windowMs = 60000 } is a fixed window per configured API key, and per remote address for anything else. Flags are --rate-limit and --rate-limit-window. Over the limit the client gets a 429 in its front's error format with retry-after, before any upstream is contacted. The limit doesn't apply to /, /api/hello, /metrics or preflights. Limits are per process, and behind a reverse proxy every request shares the proxy's address.
  • CORS: cors is a list of allowed origins, or "*"; the flag is --cors-origin (repeatable). Allowed origins get CORS headers on every reply, including errors and streams. Preflights get a 204 without an API key and outside the rate limit. Requests without an Origin header are untouched.

When requests come in, CORS preflights are answered first, then metrics counts the request, then the rate limit applies, then the API key is checked. A request refused with 429 shows up in the metrics as code="429". See docs/server.md. Resolves #52.

Failed tool results are marked by default (#169)

Chat Completions has no error flag on tool messages, so a failed tool reached the model as ordinary output. toolErrors now defaults to true: a tool_result with is_error: true gets toolErrorMarker ("[error] ") in front of its text on every OpenAI-protocol driver, including vLLM's native /v1/messages passthrough. Upstreams that take is_error natively are unaffected. toolErrorInference stays off by default, because a tool that just prints the marker would be read as a failure. See docs/settings.md. Resolves #124.

📚 Documentation


Upgrade notes

  • Failed tool results now carry a marker. If a model or prompt depended on seeing failed tool output unchanged over Chat Completions, set toolErrors: false.
  • Metrics, TLS, rate limits and CORS are opt-in. No other changes are needed.

coverage

Metric Coverage
Statements 100.0%
Branches 100.0%
Functions 100.0%
Lines 100.0%

Avg: 100.0% · ab1a862 · Node lts/*

👥 Contributors

v0.11.0

Choose a tag to compare

@cldmv-bot cldmv-bot released this 17 Nov 14:29
e0320e9

llm-translator v0.11.0 Changelog

Release Date: November 2026
Release Type: Minor
Branch: next → master


Overview

Version 0.11.0 adds three things and fixes one. context_management clear_tool_uses edits are now emulated for OpenAI-protocol upstreams, a neutral constraint (JSON Schema, regex, choice or grammar) is turned into each backend's native constrained-decoding field, and the vLLM cache_salt and OpenAI prompt_cache_key hints are mapped. The fix makes upstreams on a known hosted API bind by host name before any driver probes them.

Everything new is opt-in except the host-first binding and the clear_tool_uses edit, which only applies when a client sends one. The cache and constraint field names come from each backend's documentation and have not been run against a real server.


🐛 Bug Fixes

Host-matched upstreams bind before any driver probes (#163)

A driver can name the hosts it owns in info().hosts, or a host suffix in info().hostSuffixes. Detection now tries those drivers first, whatever their order, and binds the upstream with no request at all. Before, a hosted API such as api.openai.com was probed as vLLM (GET /version, GET /v1/models) and other self-hosted servers before its own driver got a turn. The openai, deepseek, mistral and openrouter drivers declare their hosts, azure declares its three resource suffixes, and the presets for Groq, Together, Fireworks, xAI and Gemini do the same. Every other upstream is probed in order as before. See docs/drivers.md. Fixes #158.

✨ Features

clear_tool_uses context edits (#162)

An Anthropic request carrying a clear_tool_uses_20250919 edit in context_management has it applied locally, on the converted messages, before it goes to an OpenAI-protocol backend. The reply reports what was cleared in context_management.applied_edits, streamed or not.

  • trigger (input_tokens or tool_uses, default 100000 input tokens), keep (default the 3 most recent tool uses), clear_at_least, exclude_tools and clear_tool_inputs are honoured.
  • Clearing goes from the oldest tool use forward. A cleared result becomes [Tool result cleared to save context], the call stays, and with clear_tool_inputs: true its arguments become {}.
  • Input tokens are estimated at one per four characters, so a trigger can fire somewhat earlier or later than on Anthropic.
  • An edit with parameters outside the documented shape is dropped as a whole. Compaction and unknown edit types are still dropped and recorded in state.dropped.

clear_thinking was already emulated. See docs/protocol-support.md and docs/known-limitations.md. Resolves #13.

Native constrained decoding (#164)

A neutral constraint describes one of json (a JSON Schema), regex, choice (a list of strings) or grammar. Set it per upstream with the constraint setting, or per request as constraint on the OpenAI front and metadata.constraint on the Anthropic front. The driver of the upstream turns it into the backend's own field, and the extension is never forwarded under its own name.

Backend json regex choice grammar
vLLM 0.11.0 and later, unknown version, compatible structured_outputs structured_outputs structured_outputs structured_outputs
vLLM before 0.11.0 guided_json guided_regex guided_choice guided_grammar
SGLang response_format regex regex ebnf
llama.cpp json_schema not supported grammar grammar
TGI response_format response_format response_format not supported
other OpenAI-protocol response_format not supported not supported not supported

A kind the backend cannot enforce follows unknownFields: dropped and recorded as constraint, or rejected. vLLM applies a structured-outputs constraint to the whole generation, so when a request offers tools the translator sends the tools and drops the constraint. A request sent to vLLM's own /v1/messages (nativeMessages) does not get the constraint either. The OpenAI front's passthroughFilter now lets vLLM's structured_outputs and guided_* fields, and llama.cpp's grammar and json_schema, through for those backends only. See docs/drivers.md. Resolves #14.

Prompt-cache hints (#165)

Two hints are mapped, and each goes to one backend only. Everything else about caching is unchanged: cache_control is still dropped and cache_creation_input_tokens is still always 0.

  • cache_salt (vLLM only). vLLM mixes the salt into its prefix-cache hash, so requests with different salts never share cached prefixes. It is set by the server, never by a client: the cacheSalt setting per upstream, or the cacheSalt call option of request and send, which wins. For a per-API-key salt, an authorize hook returns { ok: true, cacheSalt }. A cache_salt in a client body is never used. Other drivers drop it and record cache_salt, or reject it under unknownFields: "error".
  • prompt_cache_key (OpenAI only). Sent to the openai driver from the client's own field, metadata.prompt_cache_key on the Anthropic front, or the promptCacheKey setting for requests with none. Other drivers drop it and record it. prompt_cache_retention is not mapped.

docs/server.md has a new Prompt caching section on what is and isn't cached, with guidance on keeping prefixes stable. See docs/server.md. Resolves #15.

📚 Documentation

🔧 Dependencies

  • prettier 3.9.10 → 3.10.1 (#159), a dev dependency.

Upgrade notes

  • Upstreams on known hosts bind without probing. api.openai.com, the DeepSeek, Mistral and OpenRouter hosts, Azure OpenAI resources and the hosted-provider presets now bind to their own driver at creation with no request. Set backend on the upstream to choose another driver.
  • cache_salt is server-side only. A client cannot set it; use the cacheSalt setting, the call option or the authorize hook.
  • No other changes are needed.

coverage

Metric Coverage
Statements 100.0%
Branches 100.0%
Functions 100.0%
Lines 100.0%

Avg: 100.0% · d730c97 · Node lts/*

👥 Contributors

v0.10.0

Choose a tag to compare

@cldmv-bot cldmv-bot released this 14 Nov 17:13
5b56be2

llm-translator v0.10.0 Changelog

Release Date: November 2026
Release Type: Minor
Branch: next → master


Overview

Version 0.10.0 adds four things. Anthropic requests to vLLM can go to its own /v1/messages instead of being converted, a pool skips vLLM upstreams that aren't ready and routes to LoRA adapters, five hosted OpenAI-compatible providers (Groq, Together, Fireworks, xAI and Gemini) get drivers, and Ollama can be used through its native /api/chat protocol.

Everything is opt-in except the provider drivers, which bind by host name, and the readiness probes, which only run when a model has more than one endpoint. One behaviour change applies to upstreams with backend: "vllm" set by hand; see the upgrade notes.

None of this has been run against a real server yet. It's built from the providers' API docs and source and tested against fixtures.


✨ Features

Native /v1/messages for vLLM (#152)

vLLM 0.11.1 and later serve the Anthropic Messages API themselves. A new nativeMessages setting (true, false or "auto", default false) sends an Anthropic-front request to that endpoint with only normalisation, instead of converting it to Chat Completions and back. "auto" uses it when the detected or configured version is 0.11.1 or later, and converts when the version is unknown.

  • The system prompt is sent as one string with the billing header block removed.
  • vLLM sends no ping event, so one follows message_start.
  • Thinking blocks vLLM sends without a signature (before 0.17) get the translator's marker signature.
  • With toolErrors on, failed tool results carry the marker text, as in a conversion, since vLLM ignores is_error.
  • The upstream is called with the bearer key only; the client's x-api-key and anthropic-beta headers aren't forwarded.

A request ending in an assistant prefill, one with document blocks, and one with deferred tools (unless toolSearch is native) are still converted. An upstream that answers 404 to /v1/messages with something other than an Anthropic error is converted from then on, until the next detect(), and a fallback event is emitted once. status() shows whether an upstream is on the native endpoint. See docs/drivers.md. Resolves #31.

Skip unhealthy upstreams and tag LoRA adapters (#153)

When more than one endpoint serves a model, upstreams whose driver names a health path (vLLM: GET /health) are checked before the pool is ordered, and the ones that aren't ready go last. A lone endpoint is still tried.

  • New pool.healthInterval (milliseconds a verdict is trusted, default 10000, 0 turns probing off) and pool.healthTimeout (default 2000).
  • endpoint.status() and status() carry ready, readyReason and checkedAt. A health event with kinds down and up fires when an upstream changes state, and health.check() probes every upstream right away.
  • vLLM lists each LoRA adapter in /v1/models as a model whose parent is the base model. Adapters are kept apart in the detection facts (facts.adapters), are routable like any other model, and carry parent in model.list() and GET /v1/models on both fronts. An upstream with one base model and some adapters is still the single model unknown names fall back to.
  • A model list fetched live now updates what a vLLM upstream serves, so adapters loaded or unloaded at runtime start or stop being routable. See the upgrade notes.

See docs/drivers.md and docs/server.md. Resolves #32.

Presets for hosted OpenAI-compatible providers (#154)

groq, together, fireworks, xai and gemini drivers, bound by host name (api.groq.com, api.together.ai, api.fireworks.ai, api.x.ai, generativelanguage.googleapis.com) or chosen with backend for a proxy in front of the provider. The provider's path root is added for you, so the host alone works as the upstream url.

The provider facts sit in one table, PRESETS in src/lib/presets.mjs: the path root, the token field (max_tokens or max_completion_tokens), the reasoning field, how thinking and effort are sent, whether top_k is accepted, whether the stream takes stream_options, and the request fields the provider rejects (Groq's logprobs and logit_bias, xAI's stop and penalties on reasoning models). The driver folders are one-line wrappers over it, and settings on an upstream still overrides any value. See docs/drivers.md. Resolves #44.

Ollama's native /api/chat protocol (#155)

An api: "native" setting on an ollama upstream (default "openai", the existing /v1 path) talks to Ollama's own chat API, which exposes the controls /v1 hides: think, options and format. Both fronts are converted to and from it, streamed or not.

  • Tool arguments are JSON objects and tool results are matched by tool_name. Ollama sends no tool-call ids, so each call gets a generated one.
  • think takes the effort level when there is one, else the thinking switch. message.thinking is read back as a thinking block (or the reasoning field on the OpenAI front) and sent on assistant turns.
  • Sampling fields go into options, response_format and output_config.format become format, and images go as base64 strings.
  • The stream is newline-delimited JSON, buffered across chunks so a line split at any byte is parsed once complete.
  • tool_choice other than auto, n above 1, logit_bias and logprobs are dropped and recorded in state.dropped. The model list and embeddings stay on /v1.

Setting api: "native" on an upstream bound to a driver without a native API is an error at detection. See docs/drivers.md and docs/protocol-support.md. Resolves #35.

📚 Documentation

🔧 Dependencies

No dependency updates.


Upgrade notes

  • An upstream with backend: "vllm" and no models list now serves what the server last listed. An explicit backend skips detection, so such an upstream had no model list of its own and a model list fetched from the server was never kept. Now every successful fetch (model.list(), GET /v1/models) replaces the upstream's served models, adapters included, with what the server returned, and requests for those models route to it. An upstream with a configured models list keeps that list, and backend: "auto" upstreams get the same refresh on top of what detection found.
  • Pools probe vLLM upstreams. A model with more than one endpoint now gets a GET /health on each vLLM endpoint, at most once per pool.healthInterval. A server that answers something other than 2xx or 5xx (a proxy without that path) is left alone until the next detect(). Set pool.healthInterval: 0 to turn probing off.
  • Provider hosts change driver. api.groq.com, api.together.ai (and api.together.xyz, us-api.together.ai), api.fireworks.ai, api.x.ai and generativelanguage.googleapis.com upstreams are now bound to groq, together, fireworks, xai or gemini, where they ran on compatible. Set backend: "compatible" on the upstream to keep the old behaviour.
  • nativeMessages and Ollama's api: "native" are off unless you set them.
  • No other changes are needed.

coverage

Metric Coverage
Statements 100.0%
Branches 100.0%
Functions 100.0%
Lines 100.0%

Avg: 100.0% · 60e4777 · Node lts/*

👥 Contributors

v0.9.0

Choose a tag to compare

@cldmv-bot cldmv-bot released this 14 Nov 00:54
22a7336

llm-translator v0.9.0 Changelog

Release Date: November 2026
Release Type: Minor
Branch: next → master


Overview

Version 0.9.0 adds four drivers: Azure OpenAI, OpenRouter, Hugging Face Text Generation Inference (TGI) and LocalAI. All four used to run on the generic compatible driver at best. Azure OpenAI didn't work at all without rewriting URLs by hand. Each driver now handles its server's own URL layout, auth, reply quirks and error shapes.

To make Azure possible, a driver can now build its own request path through an optional path leaf, and api-key joins bearer and x-api-key as an auth style. Upstreams on these servers move to their new drivers automatically, and self-hosted servers get a few more startup probes. See the upgrade notes.

None of the four has been run against a real server yet. They're built from the providers' API docs and source and tested against fixtures.


✨ Features

Azure OpenAI driver (#145)

An azure driver, matched by host name (*.openai.azure.com, *.cognitiveservices.azure.com, *.services.ai.azure.com) with no request of its own. The key goes in the api-key header.

  • By default requests go to the v1 API (<endpoint>/openai/v1/chat/completions, the model named in the body, no api-version).
  • Set apiVersion on the upstream to use the classic URLs instead: <endpoint>/openai/deployments/<deployment>/chat/completions?api-version=<version>. deployment is a second new upstream field and defaults to the request's model. The model list and embeddings follow the same layout.
  • A base URL pasted with /openai or /openai/v1 on the end works too.
  • Token limits go out as max_completion_tokens. With an apiVersion dated before 2024-09-01 they go out as max_tokens, the only field those versions know.
  • A prompt rejected by the content filter (HTTP 400, code: "content_filter") keeps that code. Its message lists the categories that were filtered, with their severity. Filtered replies already got stop_details.category in v0.8.0.

Entra ID tokens aren't built in; send one through the upstream's headers. See docs/drivers.md. Fixes #43.

OpenRouter driver (#146)

An openrouter driver for openrouter.ai, matched by host name. https://openrouter.ai, https://openrouter.ai/api and https://openrouter.ai/api/v1 all work as the base URL.

  • reasoning_details (reasoning.text, reasoning.summary and reasoning.encrypted items) comes back to OpenRouter unmodified and in order on the next turn, streamed or not.
  • OpenRouter can answer HTTP 200 with an error on the choice instead of a message. That's now a real upstream error with the provider's status and error type, and in a stream it ends the stream with an error event.
  • Reasoning effort is sent as reasoning: { effort }, and Anthropic thinking with a budget_tokens (and no explicit effort) as reasoning: { max_tokens }. A reasoning object an OpenAI client sends itself is passed through as is.
  • The final usage chunk and the : OPENROUTER PROCESSING keep-alive comments are handled.

Fixes #41.

Hugging Face TGI driver (#147)

A tgi driver, detected by a strict match on GET /info. It needs router: "text-generation-router", a string model_id and version, and a numeric max_total_tokens, so Text Embeddings Inference, which answers /info as text-embeddings-router, isn't claimed. vLLM now steps aside for it.

TGI's replies are normalised for the deviations older releases are known for:

  • tool_calls sent as a single object;
  • numeric tool-call ids, and empty ids on streamed fragments, which used to open a new tool_use block per fragment;
  • tool arguments as an object instead of a JSON string;
  • the eos_token and stop_sequence finish reasons;
  • the text_completion object name.

Structured output is sent in TGI's grammar shape: a JSON schema becomes response_format: { type: "json_schema", value: { name, schema } }. A client's own grammar is accepted and folded into response_format. TGI refuses tools and a grammar together, so tools win and the grammar is dropped. Fixes #38.

LocalAI driver (#148)

A localai driver, detected by the <version> (<commit>) shape of LocalAI's GET /version, or by its /.well-known/localai.json discovery document. vLLM, which also answers /version, steps aside for it.

  • Reasoning is read from LocalAI's reasoning field (and reasoning_content) and sent back as reasoning.
  • LocalAI sets parallel tool calls per model in its YAML (function.parallel_calls) and ignores parallel_tool_calls. When an Anthropic client sets disable_parallel_tool_use, the driver keeps only the first tool call of each reply and stream. An OpenAI client's parallel_tool_calls: false is forwarded but has no effect on LocalAI.
  • repetition_penalty is sent as repeat_penalty, and LocalAI's extra sampling fields (top_k, min_p and others) are passed through.

docs/drivers.md covers parallel_calls and the case where a model emits tool calls as plain text, with the LocalAI function: settings that fix it. Fixes #39.

📚 Documentation

🔧 Dependencies

No dependency updates. @cldmv/slothlet and @cldmv/slothlet-types stay at 3.23.0.


Upgrade notes

  • Upstreams on these servers change driver. An Azure OpenAI or OpenRouter host is now bound to azure or openrouter, and a TGI or LocalAI server to tgi or localai, where they used to run on compatible (or, for TGI and LocalAI, could be taken for vLLM). Set backend: "compatible" on the upstream to keep the old behaviour.
  • Azure upstreams that rewrote URLs by hand. If you pointed an upstream at a full deployment URL to make compatible work, point it at the resource endpoint instead and set apiVersion (and deployment if it differs from the model name).
  • More startup probes. A server no earlier driver claims now also gets GET /info and GET /.well-known/localai.json, and vLLM's own check also asks for /info when /version doesn't name vLLM. The probes run in parallel and are bounded by detectTimeout. Set backend on an upstream to skip probing.
  • No other changes are needed.

coverage

Metric Coverage
Statements 100.0%
Branches 100.0%
Functions 100.0%
Lines 100.0%

Avg: 100.0% · 05d19bf · Node lts/*

👥 Contributors

v0.8.0

Choose a tag to compare

@cldmv-bot cldmv-bot released this 13 Nov 05:20
af0eb48

llm-translator v0.8.0 Changelog

Release Date: November 2026
Release Type: Minor
Branch: next → master


Overview

Version 0.8.0 adds two drivers, Mistral and LM Studio, answers an OpenAI client's n above 1 on Anthropic-protocol upstreams by fanning out to parallel requests, and fills in stop_details.category when an Azure-style content filter says which harm category it caught. It also fixes the shared stream converter so a tool call whose later chunks carry id: null stays one call.

Nothing here needs a config change. A Mistral or LM Studio upstream that ran on the generic compatible driver is now bound to its own driver, and a server no earlier driver claims gets one more startup probe, GET /api/v0/models. See the upgrade notes.


✨ Features

Mistral API driver (#139)

A mistral driver for api.mistral.ai, matched by host name with no request of its own. It is the first built-in driver to declare info().toolCallIds ({ alphabet: "a-zA-Z0-9", min: 9, max: 9 }), so history replayed from an Anthropic client, or from an OpenAI client with other ids, goes out with valid 9-character ids, and the ids Mistral returns map back to the client's originals.

seed is sent as random_seed, and tool_choice: "required" as "any". Reasoning models answer with content as a list of typed chunks; the driver turns the text chunks into content and the thinking chunks into reasoning_content, which becomes thinking blocks, streamed and not. The driver hasn't been run against the real API yet; that is tracked in #70. Fixes #42.

LM Studio driver (#140)

An lmstudio driver, detected by a strict match on LM Studio's GET /api/v0/models listing: object: "list", a non-empty data, and every entry with object: "model", a string id, arch, compatibility_type and state, and a type of llm, vlm or embeddings. An OpenAI-style /v1/models list can never pass. The probe goes out with the other startup probes and is bounded by detectTimeout. The served models are the chat models from that list, embedding models left out.

LM Studio also serves an Anthropic-compatible POST /v1/messages; set backend: "anthropic" on the upstream to use it instead of the OpenAI-style translation. docs/drivers.md covers both. Not run against a real LM Studio server yet (#70). Fixes #37.

n above 1 on Anthropic-protocol upstreams (#142)

Anthropic's Messages API has no n, so an OpenAI client asking for several choices from an Anthropic-protocol upstream used to get an error. Now send runs n ordinary single-choice sends in parallel and combines them. Each branch is a full send with n: 1, so pools, failover, retries and capture apply per branch.

  • Without streaming, the reply has every branch's choice, numbered 0 to n-1, and the summed usage.
  • With streaming, chunks go out as they arrive, re-indexed to their branch, with one final usage chunk (when stream_options.include_usage asked for it) and a single [DONE].
  • The first branch to fail ends the request and aborts the rest. Aborting the caller, or the client disconnecting, cancels every branch.
  • onUsage is called once with the combined report, and each branch is its own capture exchange.

OpenAI-protocol upstreams still get n as is. An n above 128 is a 400. The fan-out decision is made on the endpoint the router picks first; see docs/server.md and decision D-49. Fixes #26.

stop_details.category from content-filter results (#141)

A refusal used to carry stop_details.category: null every time. When the upstream reports Azure OpenAI style content_filter_results and one of hate, sexual, violence or self_harm is filtered, on the choice or on any streamed chunk, the category is now "general_harms". Jailbreak, protected-material, profanity and blocklist matches, and every upstream that reports no filter details, still give null. The other Anthropic categories (cyber, bio, frontier_llm, reasoning_extraction) are never produced. The reasoning is recorded as decision D-50. Fixes #123.

🐛 Bug Fixes

Streamed tool calls with id: null stay one call (#140)

The OpenAI → Anthropic stream converter treated a tool-call chunk with id: null as a new call, so a server that sends the id only on the first fragment (LM Studio does) produced one tool_use block per fragment. A null id is now treated like a missing one and continues the current call, with or without an index.

📚 Documentation

🔧 Dependencies

  • @napi-rs/canvas 1.0.13 → 1.1.1 (#133). Canvas 1.1 draws text with its portable FreeType renderer by default, so text in rendered PDF page images can look slightly different from before.
  • Dev dependencies: @cldmv/jsonv 1.1.5 → 1.2.2, @cldmv/eslint-plugin-jsonv 1.0.17 → 1.1.2, @cldmv/prettier-plugin-jsonv 1.1.4 → 1.2.2 and @cldmv/vitest-runner 1.5.6 → 1.6.1 (#133).
  • @cldmv/slothlet and @cldmv/slothlet-types stay at 3.23.0; 3.24.0 is skipped.

Upgrade notes

  • Mistral and LM Studio upstreams change driver. An upstream on api.mistral.ai is now bound to mistral, and an LM Studio server to lmstudio, where both used to run on compatible. Set backend: "compatible" to keep the old behaviour.
  • One extra startup probe. A server no earlier driver claims now also gets GET /api/v0/models. It runs in parallel with the other probes and only binds LM Studio's own listing. Set backend on an upstream to skip probing altogether.
  • n above 1 costs n upstream calls. On an Anthropic-protocol upstream it is billed and rate-limited as n requests.
  • No other changes are needed.

coverage

Metric Coverage
Statements 100.0%
Branches 100.0%
Functions 100.0%
Lines 100.0%

Avg: 100.0% · 3546c34 · Node lts/*

👥 Contributors

v0.7.0

Choose a tag to compare

@cldmv-bot cldmv-bot released this 08 Nov 22:16
ec6f299

llm-translator v0.7.0 Changelog

Release Date: November 2026
Release Type: Minor
Branch: next → master


Overview

Version 0.7.0 detects Anthropic-protocol servers that aren't on api.anthropic.com, lets a driver declare a tool-call id format the translator rewrites ids into, reports embeddings usage through onUsage, and makes capture record failed-over attempts and embeddings. It also adds npm run test:dist, which runs the full suite against the built dist/.

Nothing here needs a config change. The one difference an existing setup can see is at startup: a server no earlier driver claims now also gets a GET /v1/models with Anthropic headers. See the upgrade notes.


✨ Features

Detect Anthropic-protocol servers by probe (#134)

An Anthropic-compatible server on any host used to need backend: "anthropic". Now a server that no earlier driver claims is also asked for GET /v1/models with anthropic-version and the upstream's key as x-api-key. It is bound to the anthropic driver only when the answer is an Anthropic model list: no object key, a boolean has_more, first_id/last_id naming the first and last entry, and every entry with type: "model", a string id and a string display_name. An OpenAI-style list can never pass, and a server that answers with the OpenAI shape whatever headers it gets stays on the generic driver. The served models come from that list.

api.anthropic.com is still matched by host name with no request. The probe goes out together with the other startup probes, so it adds one request but no round trip. A server that requires a key and has none configured answers 401 and isn't detected; set apiKey or backend: "anthropic". Fixes #54.

Tool-call ids rewritten for strict upstream formats (#135)

Some backends only accept tool-call ids in a fixed format (Mistral wants nine alphanumeric characters, for example). A driver can now declare info().toolCallIds = { alphabet, min?, max? }, and the translator rewrites the client's ids into that format in both directions: an Anthropic client to an OpenAI-style server, and an OpenAI client to an Anthropic-protocol upstream. Ids the upstream returns are mapped back to the client's originals, streaming included.

There is no state between requests: an id that already fits passes through unchanged, and any other id becomes a SHA-256 hash of itself written in the allowed alphabet, so the same id maps the same way on every request and a call always matches its result. Conforming ids are reserved first, and collisions are rehashed with a counter. A format with too few free ids for the conversation fails with invalid_request. No built-in driver declares a format yet; the Mistral driver is #42. Fixes #20.

Embeddings usage through onUsage (#137)

api.embeddings.send and the /v1/embeddings route now report usage like chat does. Every reply, including error replies, has the non-enumerable usage promise, and onUsage is called once, with the authorize context on the server route. The report has front: "openai" and stream: false. input_tokens is the reply's prompt_tokens and the other token fields are null, and a reply without usage reports reported: false. Errors carry reason: "upstream_error", "aborted" or "error". Fixes #121.

Capture records failed-over attempts and embeddings (#136)

capture used to record only the endpoint that answered. Now every attempt of a request gets its own record, and the records share an exchange id (the first attempt's id) and a 1-based attempt. An attempt the request moved on from also has failover with the reason. Its response is the 5xx reply, or { error } for an unreachable upstream. A final failure that becomes the reply is recorded too, and embeddings are captured with the same header redaction. With capture off, failover behaves as before and no extra body is read. Fixes #122.

🔧 CI & tooling

test:dist and test:cjs (#126)

npm run test:dist builds the package and runs the whole suite against dist/ instead of src/, then the CJS entry tests. npm run test:cjs runs only the CJS tests, which load the CJS entry point. npm test runs the suite and then test:cjs, and ci:coverage now runs coverage and then test:dist.

📚 Documentation

🔧 Dependencies

Dev dependencies only:

  • eslint 10.13.0 → 10.14.0 (#129)
  • vitest and @vitest/coverage-v8 5.0.5 → 5.1.0 (#128)
  • fastify 5.12.7 → 5.12.8 and @cldmv/fix-headers 2.2.2 → 2.2.6 (#130)

Upgrade notes

  • One extra startup probe. A server no earlier driver claims now also gets GET /v1/models with anthropic-version and its key as x-api-key. It runs in parallel with the other probes and only binds a real Anthropic model list. Set backend on an upstream to skip probing altogether.
  • Capture records have new fields. Records now carry exchange and attempt, a failed-over attempt also carries failover, and a request that failed over writes more than one record. A tool that reads capture files should group them by exchange.
  • Third-party drivers: info().toolCallIds is optional. Without it, ids are passed through as before.
  • No other changes are needed.

coverage

Metric Coverage
Statements 100.0%
Branches 100.0%
Functions 100.0%
Lines 100.0%

Avg: 100.0% · 0be57c1 · Node lts/*

👥 Contributors

v0.6.0

Choose a tag to compare

@cldmv-bot cldmv-bot released this 08 Nov 05:12
e11cb04

llm-translator v0.6.0 Changelog

Release Date: November 2026
Release Type: Minor
Branch: next → master


Overview

Version 0.6.0 adds four things: POST /v1/embeddings forwarding, sampling extensions such as min_p and repetition_penalty for clients that cannot send them, strict tool use in both directions with a schema check, and a capture option that records exchanges for debugging.

capture and sampling are off until configured, and embeddings is a new route. The one change that can affect existing requests is strict: it now reaches Anthropic-protocol upstreams from Chat Completions clients, a strict tool whose schema is outside the strict subset is sent without strict and with a warning, and under unknownFields: "error" it is a 400. See the upgrade notes.


✨ Features

Embeddings pass-through (#116)

POST /v1/embeddings on the OpenAI front forwards the request to an OpenAI-protocol upstream that lists the model, and api.embeddings.send(body, { signal, headers }) does the same as a library call. The body and reply pass through unchanged, except that model becomes the upstream's name for it after aliases. Pools and failover work as for chat, apiKey and authorize apply, and errors use the OpenAI shape.

Anthropic-protocol endpoints in a pool are skipped, and a model served only by them gets a 400 embeddings_unsupported. A model no upstream lists gets a 404 model_not_found, because embeddings have no single-model fallback. Usage is not reported to onUsage yet (#121). Resolves #28.

Sampling extensions per upstream and per request (#115)

Anthropic requests have no field for min_p, repetition_penalty and similar, so there was no way to send them. The new sampling setting adds them to requests for an OpenAI-protocol upstream, for example { "min_p": 0.05, "repetition_penalty": 1.1 }, and an Anthropic client can set them per request in metadata.sampling, which wins over the setting. Fields the request already carries are kept.

seed, frequency_penalty and presence_penalty go to every OpenAI-protocol upstream, and the rest only to a driver that lists the field in info().fields. Names are the vLLM ones; the llama.cpp driver sends repetition_penalty as repeat_penalty. A field the driver does not accept follows unknownFields: dropped, or rejected with unsupported_field. metadata.sampling is removed before a request goes to a native Anthropic upstream. Resolves #60.

strict tool use end to end (#117)

strict now round trips: Chat Completions to Anthropic maps function.strict to the tool's strict, which used to be lost, and the other direction already forwarded it.

Schemas of strict tools are checked against the strict subset: objects need additionalProperties: false and every property in required, and keywords such as allOf, not, if/then/else and patternProperties are flagged. A tool whose schema is outside the subset is sent without strict, with a warning event (kind: "strict"), so the request still goes through but that tool isn't enforced (#125). With unknownFields: "error" the request fails with unsupported_content instead.

A new driver flag, info().strict, says whether a backend can enforce strict tool use. It is true for anthropic, openai and vllm (where enforcement depends on --tool-strict-level). A strict tool sent to any other backend gets a warning that it is not enforced. These flags follow what each server documents; confirming them against real servers is open in #120. Resolves #57.

Capture for debugging (#118)

The capture config option records the exchanges that pass through send and the server, for troubleshooting a new backend. Set it to a directory for one JSON file per exchange, or to a callback that receives each record. It is off by default. A record holds the client request, the request converted for the upstream and the upstream's reply. Streams keep the upstream's raw SSE text and the events sent to the client, each capped at 1,000,000 characters.

The value of any header whose name contains key, token, secret, auth, cookie or password is replaced by [redacted]. Bodies are recorded as they are, so treat the files as sensitive: they contain prompts and completions. Only the endpoint that answered is recorded, and embeddings are not captured; both are open in #122. A capture that cannot be written is reported as a warning event with kind: "capture" and never changes the reply. Resolves #53.

🔧 CI & tooling

  • Corrected a ci.yml comment that said type checking runs inside the coverage-badge job. It runs as its own 🔎 Type Check job.

📚 Documentation

🔧 Dependencies

No dependency updates.


Upgrade notes

  • strict now reaches Anthropic-protocol upstreams from Chat Completions clients. A client that sets function.strict gets it enforced there for schemas in the strict subset, where it used to be dropped.
  • Strict tools outside the subset lose strict. They are sent without it, with a warning event (kind: "strict"). From Anthropic clients to OpenAI-protocol upstreams this is a change: those tools used to be forwarded with strict as they were. With unknownFields: "error" they are a 400 unsupported_content, so check strict schemas before upgrading if you run with that setting.
  • Third-party drivers: info().strict is optional. A driver without it gets a warning for each request that still carries a strict tool after the subset check, so set it to true if the backend enforces strict tool use.
  • No other changes are needed. The new sampling and capture options have no effect until set.

coverage

Metric Coverage
Statements 100.0%
Branches 100.0%
Functions 100.0%
Lines 100.0%

Avg: 100.0% · 893729c · Node lts/*

v0.5.0

Choose a tag to compare

@cldmv-bot cldmv-bot released this 06 Nov 14:01
3d7744f

llm-translator v0.5.0 Changelog

Release Date: November 2026
Release Type: Minor
Branch: next → master


Overview

Version 0.5.0 makes everything the library logs available to the application as events, and makes logging itself opt-in. It also sends OpenAI's response_format: { type: "json_object" } to Anthropic upstreams instead of dropping it, falling back to a system-prompt instruction by itself when an upstream refuses the schema form, and fills in Anthropic's stop_details for refusals.

The one behaviour change to look at: used as a library, the translator no longer prints warnings to the console unless logs is set. The CLI still prints them by default.


✨ Features

Events for everything the library logs, and opt-in logging (#111)

translator.events.on(name, fn) subscribes to an event and returns a function that removes the listener; translator.events.off(name, fn) removes it by reference. An onEvent callback in the config is registered before startup, so it also sees the startup events (the detection run and the PDF libraries loading). The general events are log, warning and error; the specific ones are fallback, failover, effort, dropped, detect and usage. One occurrence fires its specific event, then the general events that fit, then "*", all with the same { type, kind?, level, message, time, ...details } payload. Listeners only observe: one that throws or rejects is caught and never affects a request. events: false turns events off.

Logging is a separate switch and is off by default. logs: true prints warnings and errors to the console, and logs: "<path>" (relative or absolute) appends every event as a JSON line to that file. Server mode also passes events to the Fastify logger. The CLI keeps console output on by default and gains --logs [path]; --silent is the same as logs: false. The full catalogue is in docs/events.md. Resolves #109.

response_format: json_object to Anthropic, with a self-healing fallback (#108)

Anthropic has no JSON mode, so json_object used to be dropped on the way to an Anthropic-protocol upstream. The new jsonObject setting picks how it is sent: "schema" (the default) sends output_config.format as a json_schema with the schema { "type": "object" }, "instruction" adds "Respond with a single valid JSON object and nothing else." to the system prompt, and "drop" keeps the old behaviour.

Not every upstream accepts structured outputs, so with jsonObjectFallback (on by default) a 400 or 422 answer to the schema form is retried once with the instruction. When the retry works, that upstream and model are remembered until the process ends, and a fallback warning carries the upstream's status and error message, so known upstreams can get their own setting later. When the retry fails too, the original error is returned and nothing is remembered. A json_schema the client sent itself is never retried. Resolves #59.

stop_details for refusals (#107)

A content_filter finish now produces stop_reason: "refusal" with stop_details: { type: "refusal", category: null, explanation }, using the upstream's refusal text as the explanation, both streamed and not. A streamed refusal with no answer text now comes through as the text block, the same as the non-streaming reply. Resolves #21.

📚 Documentation

🔧 Dependencies

No dependency updates.


Upgrade notes

  • Library warnings are quiet by default. Set logs: true to get the console warnings back, or logs: "<path>" to write them to a file. Subscribing to warning or log is the other way to see them, and events fire whether logging is on or not.
  • silent is deprecated. It still works and means logs: false; logs wins when both are set.
  • json_object requests to an Anthropic-protocol upstream now carry a schema instead of being dropped. Set jsonObject: "instruction" on an upstream known to lack structured outputs to skip the failed first attempt, or jsonObject: "drop" for the old behaviour.

coverage

Metric Coverage
Statements 100.0%
Branches 100.0%
Functions 100.0%
Lines 100.0%

Avg: 100.0% · ed7053e · Node lts/*

👥 Contributors

v0.4.0

Choose a tag to compare

@cldmv-bot cldmv-bot released this 05 Nov 14:13
e624e40

llm-translator v0.4.0 Changelog

Release Date: November 2026
Release Type: Minor
Branch: next → master


Overview

Version 0.4.0 lets the translator know which reasoning efforts each model honours, and lets an alias pin effort and thinking, so one local model can sit behind Claude Code's model tiers at different depths. It also maps the non-standard finish reasons some servers send, makes the tool-error marker configurable (with opt-in inference in the other direction), and takes the keep-alive interval from the upstream a request goes to.

The Node.js floor drops from 24 to 22.15. Nothing is removed and every existing config keeps working; the one behaviour change to look at is that fatal finish reasons now surface as errors instead of an apparently finished answer.


✨ Features

Per-model effort levels and aliases that pin effort and thinking (#99)

A new efforts config key lists the levels a model honours, by model name after aliases: a list such as ["low", "high", "max"], or { "levels": [...], "default": ... }. For a listed model on an OpenAI-protocol upstream, a listed effort is sent unchanged and any other snaps to the nearest listed level (a tie goes to the higher one) with a warning. A request with no effort gets the model's default: a level, null to send nothing, or by default the listed level nearest medium. The vLLM driver's max/xhigh → high clamp and the medium default no longer apply to listed models; models without a list behave as before.

An alias can now be { "model": "...", "effort": "max", "thinking": false } as well as a plain string. Pinned values override the request's, and the effort still goes through the model's list. GET /v1/models lists aliases (alias_of) and adds efforts, default_effort, effort and thinking to the entries they apply to, in both the Anthropic and OpenAI shapes. Resolves #80, resolves #81; the reasoning is in D-48 in docs/decisions.md.

Non-standard finish reasons (#102)

Finish reasons outside OpenAI's set used to all become end_turn. Now model_length maps to max_tokens, eos and eos_token to end_turn, and abort, aborted, error and insufficient_system_resource to an error: a single error event in a stream, or a 502 (503 for insufficient_system_resource) for a non-streaming reply. Resolves #27.

Configurable tool error marker and is_error inference (#103)

toolErrorMarker (default "[error] ") sets the text toolErrors puts in front of a failed tool result. The new toolErrorInference setting (off by default) works the other way: an OpenAI tool message whose text starts with the marker becomes a tool_result with is_error: true, with the marker removed. toolErrors stays off by default for every driver. Resolves #58.

Per-upstream pingInterval (#104)

The server now takes pingInterval from the routed upstream's settings, over the global value as for every other setting, and uses it for the early-commit wait as well as the keep-alives. In a round robin pool it is the member the request goes to; a failover to another member keeps the first member's interval. Resolves #55.

🔧 Compatibility

Node.js 22.15 floor (#105)

engines.node is now >=22.15.0 (was >=24.0.0), and the build targets node22. unpdf 2 declares Node.js 24, but nothing the translator calls in it needs 24: PDF.js polyfills the newer built-ins on 22 and 24, and the one it doesn't is only reached by XFA images and the viewer's signature editor. npm prints an engine warning for unpdf on Node.js 22; text extraction and page rendering work. Node.js 26 is the fastest. Developing the package needs Node.js 22.22 (devEngines), and CI's minimum matrix version follows it.

🔧 CI & tooling

  • The tests set the llm-translator-dev condition with devCondition() from @cldmv/configs (#100). The old test.nodeOptions entry is not a Vitest option and did nothing; the resolve and SSR conditions it sat next to are now set by the same helper, along with test.execArgv for the workers.

📚 Documentation

🔧 Dependencies

  • Dev only: @cldmv/configs ^1.3.0 → ^1.4.0, for devCondition() (#100).

Upgrade notes

  • No config changes are needed; string aliases and models without an efforts list behave as before.
  • A response that ends with abort, aborted, error or insufficient_system_resource is now an error rather than a normal end_turn message. Clients that retried on errors will now retry these too.
  • Node.js 22.15 or newer is enough to run the package.

coverage

Metric Coverage
Statements 100.0%
Branches 100.0%
Functions 100.0%
Lines 100.0%

Avg: 100.0% · d69c427 · Node lts/*

👥 Contributors