Repository navigation
v0.12.0
llm-translator v0.12.0 Changelog
Release Date: November 2026
Release Type: Minor
Branch: next → master
Overview
Version 0.12.0 is about running the server in production. It adds an optional Prometheus /metrics endpoint, and TLS, per-key rate limits and CORS for the built-in server. All four are off unless you turn them on, so an existing deployment behaves as before.
One default changes: a failed tool result sent to a Chat Completions upstream is now marked with toolErrorMarker out of the box. Set toolErrors: false to keep the old behavior.
✨ Features
Prometheus metrics (#168)
GET /metrics serves Prometheus text format (0.0.4) with no extra dependency. Turn it on with --metrics, LLM_TRANSLATOR_METRICS=1, "server": { "metrics": true } in the config file, or the metrics: true option of server.start and server.plugin. When it's off, /metrics is an unknown route. The endpoint uses the same apiKey / authorize authentication as every other route.
| Metric | Type | Labels |
|---|---|---|
llm_translator_requests_total |
counter | front, route, driver, code |
llm_translator_request_duration_seconds |
histogram | front, route, driver, code |
llm_translator_tokens_total |
counter | driver, type |
llm_translator_errors_total |
counter | kind |
llm_translator_upstream_up |
gauge | upstream, driver |
Labels are bounded: route patterns, never URLs, and never models, request ids or keys. A client that disconnects is counted as code="499" and not as an error. Counters are in memory per server. Requests to unmatched paths and the scrapes themselves are not counted. See docs/server.md. Resolves #51.
TLS, per-key rate limits and CORS (#170)
Each is a server.start option and a server key in the config file, and each has CLI flags and environment variables. cors and rateLimit also work on the Fastify plugin.
- TLS:
tls: { key, cert, ca, passphrase }takes file paths and serves HTTPS. Flags are--tls-key,--tls-cert,--tls-caand--tls-passphrase. A missing or unreadable file, orkeywithoutcert, fails start-up withinvalid_config. Files are read once, with no reload. - Rate limit:
rateLimit: { requests, windowMs = 60000 }is a fixed window per configured API key, and per remote address for anything else. Flags are--rate-limitand--rate-limit-window. Over the limit the client gets a 429 in its front's error format withretry-after, before any upstream is contacted. The limit doesn't apply to/,/api/hello,/metricsor preflights. Limits are per process, and behind a reverse proxy every request shares the proxy's address. - CORS:
corsis a list of allowed origins, or"*"; the flag is--cors-origin(repeatable). Allowed origins get CORS headers on every reply, including errors and streams. Preflights get a 204 without an API key and outside the rate limit. Requests without anOriginheader are untouched.
When requests come in, CORS preflights are answered first, then metrics counts the request, then the rate limit applies, then the API key is checked. A request refused with 429 shows up in the metrics as code="429". See docs/server.md. Resolves #52.
Failed tool results are marked by default (#169)
Chat Completions has no error flag on tool messages, so a failed tool reached the model as ordinary output. toolErrors now defaults to true: a tool_result with is_error: true gets toolErrorMarker ("[error] ") in front of its text on every OpenAI-protocol driver, including vLLM's native /v1/messages passthrough. Upstreams that take is_error natively are unaffected. toolErrorInference stays off by default, because a tool that just prints the marker would be read as a failure. See docs/settings.md. Resolves #124.
📚 Documentation
- NEW: docs/changelog/v0/v0.12.0.md: this changelog.
- docs/server.md has new Prometheus metrics and TLS, rate limits and CORS sections. docs/settings.md, docs/known-limitations.md, docs/drivers.md, docs/protocol-support.md and the feature inventory are updated to match.
Upgrade notes
- Failed tool results now carry a marker. If a model or prompt depended on seeing failed tool output unchanged over Chat Completions, set
toolErrors: false. - Metrics, TLS, rate limits and CORS are opt-in. No other changes are needed.
| Metric | Coverage |
|---|---|
| Statements | 100.0% |
| Branches | 100.0% |
| Functions | 100.0% |
| Lines | 100.0% |
Avg: 100.0% · ab1a862 · Node lts/*