v0.11.0

0.11.0 (2026-11-03)

This release expands the model catalog and client compatibility, with fixes across extraction, generation, and self-hosted deployment.

⚠ BREAKING CHANGES

• rust: adopt maintained Candle ZIP 9 backport (#765)

New models

• google: google/embeddinggemma-2.
• ibm-granite: ibm-granite/granite-speech-5.0-2b.
• perplexity-ai: perplexity-ai/pplx-decider-flash-v1.2, perplexity-ai/pplx-decider-v1.1-27b.
• VAGOsolutions: VAGOsolutions/SauerkrautLM-MoEdernColBERT-115M.
• xiaomi-research: xiaomi-research/Xiaomi-OCR-0.

Features

• EmbeddingGemma 2 provides text-only embeddings through the transformers5 bundle, requiring Transformers 5.19 or later.
• Granite Speech 5.0 adds speech-to-text extraction from audio.
• Perplexity Decider answers typed choice and score questions from text-only inputs.
• SauerkrautLM MoEdernColBERT provides text multivector embeddings and scoring with an FP32 serving profile.
• Xiaomi OCR extracts documents, text, tables, and formulas from images, using SGLang by default.
• The gateway and single server accept Anthropic Messages clients at /v1/messages; token counting is not served. System One and OpenAI Decisions compatibility includes Python and TypeScript client support.
• Opt-in runtime LoRA supports standard BERT and unquantized XLM-R rerankers, plus GLiNER2 entity extraction; cold entity-adapter requests return 503 LORA_LOADING and must be retried after readiness.
• Self-hosted servers support opt-in token authentication, with default-off Helm token generation for gateways and workers.

Bug fixes

• MADLAD loads artifacts from Hugging Face's shared blob store and removes Moses entity escaping; French elision is rejoined only for French targets.
• Model fixes correct BERT scoring on Apple Silicon CPU, BGE Flash embedding addition order, independent GTE code revisions, and ColBERT unknown-token handling; Snowflake Arctic Embed loads without xformers, and Qwen3.6-27B's H100 profile fits one H100 80GB.
• GLiNER preserves word pieces and counts long-word tokens once across windows; GLiNER2 preserves repeated mentions and isolates malformed results, with corrected document-window usage. Docling text and Markdown exports retain picture text.
• Chat fixes preserve MLX thinking-model answers, keep long tool calls alive, accept OpenRouter's final usage events, and render single-server tools as sent; upstream input refusals return 400, and other upstream failures expose safe reason codes.
• Grammar requests retain explicitly named remote profiles, and queued native prompts reach chat-only upstreams as one user turn; typed decisions resolve aliases and preserve refusal and malformed-output errors.
• LoRA fixes preserve selectors and profile defaults across clients, roll back failed attachment, resolve declared adapters through configured caches, support partial QKV adapters in SGLang, and release GPU slots on unload.
• The CLI honors environment cache and Hub-fallback settings; cluster downloads stay within their destination, and bundle images constrain compatible dependencies to uv.lock.
• Whisper restores SDPA after word-timestamp requests; sidecar scheduling separates long audio from clips, idle workers wake for admitted work, and the model-loading Prometheus metric retains its KEDA-compatible name.

Performance improvements

• GLiNER loads faster by constructing encoders on the target device, falling back to CPU construction if GPU memory is insufficient.

Upgrade notes

• Candle now uses ZIP 9, which tries UTF-8 filenames before CP437 regardless of the filename flag. Some NPZ/PTH archive names therefore decode differently; review affected archives before upgrading. Safetensors is unaffected.
• LoRA requests must select adapters declared by the chosen profile or receive 400. Move experimental CrossEncoder lora_paths profiles to sie_server.adapters.cross_encoder.runtime_lora:CrossEncoderLoraAdapter; unknown numeric configuration fields and base-mutating adapter options are rejected.
• Local Sentence Transformers custom modules and tokenizers require explicit trust_remote_code: true; the supported loader is pinned to Sentence Transformers 5.7.0.
• Chat tool names must contain 1–64 ASCII letters, digits, underscores, or hyphens, otherwise requests receive 400. Single-node remote chat also enforces declared tools and tool-choice constraints.
• Qwen3-Embedding 0.6B and 8B CUDA defaults switch to SGLang. Use explicit :flash profiles for CPU/Mac or existing Flash deployments; keep one backend per index or rebuild vectors when migrating.
• BGE Flash profiles now use FP16 defaults, while the GLiNER PII default profile uses full precision.
• Helm rejects bundled dependency credential-key collisions and mismatched trust-manager Helm-TLS namespaces. Separate credential keys; use the release namespace or cert-manager certificate generation for trust-manager.
• GLiNER2 prompt admission now counts compiled prompts rather than estimates, which can change schema acceptance; Python and TypeScript SDKs reject incomplete generation streams.
• /ws/status now returns empty user and working_dir, and 0 for pid. These fields are deprecated and will be removed no earlier than 0.13.0.

All changes (160)

Features

• adapters: add explicit BERT CrossEncoder runtime LoRA (#775) (b35cfc6)
• adapters: add explicit GLiNER2 entity runtime LoRA (#789) (dca37b7)
• adapters: extend explicit runtime LoRA to XLM-R rerankers (#787) (0fd7c9a)
• adapters: make CrossEncoder runtime LoRA explicitly opt in (fc7eeac)
• adapters: support Perplexity decision checkpoints (#791) (0521464)
• api: report locally ready LoRA adapters (c432d4c)
• api: serve declared runtime LoRAs on direct generation routes (a1f82eb)
• api: support System One and OpenAI Decisions clients (9f22374)
• cross-encoder: add declared runtime LoRA lifecycle (3bc2c41)
• examples: repeat the published benchmark numbers from one client (#880) (12376be)
• examples: replay the published fixed-rate performance records from one client (#881) (31ecf76)
• examples: run the OpenCode example on one GPU with Qwen3.8-27B-FP8 (#756) (f886c03)
• examples: run the OpenCode example on self-hosted SIE with an OpenRouter upstream (#735) (1760536)
• examples: train cost-aware routers on BBEH outcomes (#795) (b44c475)
• examples: translate one product string with Hy-MT2 1.8B (#862) (4689199)
• examples: try a local System One model with the TypeSafe SDK (1b11f4c)
• examples: validate the published-benchmarks example against two records (#885) (142a6e6)
• gateway: add OpenAI-compatible decisions endpoint (188333a)
• gateway: count /v1/systemone under the extract metrics label (cea23d9)
• gateway: serve the Anthropic Messages API on /v1/messages (#753) (1f9e0ff)
• gateway: serve the System One-compatible /v1/systemone (562b8e6)
• gliformer: add opt-in exact encoder CUDA graphs (#796) (a206d16)
• helm: add extraInitContainers and extraVolumes to worker pods (#809) (b94c193)
• helm: generate gateway and worker tokens behind default-off values (#750) (2e6350e)
• models: add ibm-granite/granite-speech-5.0-2b for speech-to-text extraction (3724719)
• models: add perplexity-ai/pplx-decider-flash-v1.2 and serve format-2 joint-protocol checkpoints (93efb56)
• models: add VAGOsolutions/SauerkrautLM-MoEdernColBERT-115M (d278019)
• models: onboard xiaomi-research/Xiaomi-OCR-0 (#875) (96a905c)
• models: report description and release_date in /v1/models (12276f5)
• sdk: add system_one to the Python and TypeScript clients (85b72f8)
• sdk: report credits with up to six decimals in optional string fields (#820) (f27d528)
• server: add a System One-compatible POST /v1/systemone (3fad998)
• server: add a text adapter and catalog entry for embeddinggemma-2 (#874) (78c5314)
• server: add LightOnOCR-3 Markdown output profile (0e3bdea)
• server: add OpenAI Decisions compatibility for text models (5cebd87)
• server: add opt-in bounded batch progress (#806) (63d0a4b)
• server: add opt-in runtime LoRA for native decision models (81ab717)
• server: add opt-in token auth and warn about exposed servers (#751) (2d56a66)
• server: add SIE_TELEMETRY_ENABLED and announce the 0.13.0 opt-in default (#747) (09b62a3)
• server: forward images and model chat-template kwargs to OpenAI-compatible upstream chat (#866) (5383e6c)
• server: let the fake engine answer chat with scripted tool calls, and run the agent-loop check in CI (#739) (055c5bd)
• server: load declared SGLang LoRAs on demand (d5e024a)
• server: release SGLang embeddings and unify Qwen3 defaults (#845) (81dec29)
• server: serve the Anthropic Messages API on sie-server (#815) (b80a029)
• server: support runtime LoRA for native decision models (7f0703b)
• sglang: load declared LoRA adapters on demand (6ae6f6c)

Bug Fixes

• address the follow-up findings from the #811 auth review (#818) (e566b05)
• api: check tool names on every chat route and report an upstream refusal as a 400 (#737) (dadd7a0)
• api: normalize LoRA request selectors across clients and gateways (23204d8)
• api: preserve LoRA selector semantics across wire encodings (81faec6)
• api: refresh model metadata and scope usage checks (24d4d0f)
• api: tell the caller why a remote upstream refused, without naming it (#744) (f52781a)
• benchmarks: account for detached resident LoRA tensors (f06f414)
• benchmarks: bind worker waits to qualification limits (a2932d8)
• benchmarks: bound cross-encoder evidence size (445c5b3)
• benchmarks: fail closed on final cleanup reporting errors (56bd852)
• benchmarks: observe direct cross-encoder forwards (7919cfd)
• benchmarks: preserve shutdown after process inspection errors (c3017d5)
• benchmarks: retain cleanup evidence on inspection errors (9fc2ce5)
• benchmarks: retain lightweight response evidence safely (84e2b18)
• bench: preserve empty LoRA slot evidence (86d0442)
• bench: validate runtime LoRA profile setup (f97830f)
• bundles: bind saved bundle locks to third-party root versions (#886) (fb2fac2)
• bundles: require transformers 5.19 in the transformers5 bundle (6a0d7b6)
• cache: confine cluster downloads to their destination (0c34750)
• cli: honor the configured Hub fallback policy (17a88bb)
• cli: keep cluster cache URLs out of startup output (10ea83b)
• cli: preserve configured cache and Hub fallback policy (#828) (a52579f)
• count each temporary token link once and correct the repeated-Authorization warning (#811) (ad4af56)
• cross-encoder: detach resident adapters from base execution (ac29fa1)
• ctranslate2: rejoin French elision only for a French target (4488cc0)
• deps: patch ctutils and Tokio security advisories (3415133)
• deps: update Rust security locks (#846) (38ef58d)
• deps: update rustls for October security advisories (#734) (bb8bdfe)
• deps: update worker TLS locks for emSign advisories (d84f169)
• deps: update workspace TLS locks for emSign advisories (e534137)
• examples: itemize router costs and validate Cloud fee labels (a8d14bb)
• examples: let the agent-loop check wait out a busy endpoint (#743) (461c5a5)
• gateway: keep grammar-constrained requests on a named remote profile (#867) (335021a)
• gateway: keep typed decision not-found errors indistinguishable (f124a99)
• gateway: report invalid LoRA selector fields (b4f642e)
• gateway: resolve configured aliases for typed decisions (27eecf0)
• gliner2: count compiled inference prompts (#794) (f3b34e2)
• gliner2: support span extraction and entity LoRA on 2.0 (#799) (98ece06)
• helm: reject mismatched trust-manager TLS namespaces (#783) (25bd316)
• helm: require worker auth capability before startup (bcd15f4)
• helm: separate bundled dependency credential keys (#782) (ac600a5)
• honor offline cache policy for decision checkpoints (4e8d7ba)
• keep chat streams alive through long tool calls (#746) (00d7d73)
• lora: integrate profile-scoped selector corrections (37cba87)
• lora: preserve profile defaults and report selector errors (a08f9af)
• lora: resolve adapters through configured artifact caches (#833) (e251cac)
• models: keep list metadata on the System One models only (f19048f)
• models: restore SGLang KV capacity for Mistral embedders and GLM-OCR (#853) (be5501a)
• models: undo Moses entity escaping in MADLAD-400 output (89f5cc5)
• narrow nullable CrossEncoder models at load time (6466ffd)
• refresh LoRA recency after worker selection (fc3f4f4)
• rust: adopt maintained Candle ZIP 9 backport (#765) (175d226)
• score: honor profile LoRA selections (4a92dc1)
• sdk: reject incomplete generation streams (118ba91)
• sdk: reject incomplete Python generation streams (8ff968c)
• sdk: reject truncated TypeScript generation streams (f1bdaf9)
• sdk: retain encode timing and call-local trace context (#733) (7eb2f08)
• security: enforce explicit trust for local ST modules (#803) (71b0c54)
• server: address the run-2 findings of the #818 auth review (#821) (9102b11)
• server: admit explicit typed-decision LoRA adapters (788faf7)
• server: admit serving artifacts from the Hugging Face shared blob store (#812) (53ba100)
• server: align BGE Flash profiles with FP16 default (975d530)
• server: align mapped BERT weights before Apple CPU inference (#792) (eebce1d)
• server: allow opting out of TF32 at startup (#825) (c296b96)
• server: constrain bundle image installs to uv.lock (#653) (e920e64)
• server: count GLiNER long-word tokens once across windows (8f36b97)
• server: declare the H100 class NV-Embed-v2 needs (#817) (271a5a9)
• server: fit the Qwen3.6-27B h100 NEXTN profile on one H100 (#816) (2af9d8c)
• server: forward independent GTE remote-code revisions (755058c)
• server: guard OpenCLIP before model construction (0062790)
• server: guard SGLang parent preprocessing (166f506)
• server: guard Transformers 4 model loading (ae3796b)
• server: keep the visible answer of a thinking model on the MLX chat path (#736) (a5d1076)
• server: load only the LoRA adapters a model declares (#752) (d183404)
• server: load snowflake-arctic-embed-m-v2.0 without xformers (#813) (c03b06c)
• server: log why a remote chat call failed, and warn when an upstream key is unset (#738) (cd21333)
• server: match BGE Flash native embedding addition order (fab8afd)
• server: meter compiled GLiNER2 document windows (#797) (b68fe36)
• server: normalize partial QKV LoRA adapters for SGLang (922e62a)
• server: parse LoRA request aliases consistently (b103712)
• server: pin OpenSearch GTE model code independently (b350a98)
• server: pin supported sentence transformers loader (11d3802)
• server: preserve decision error envelopes for malformed outputs (5d8f4bc)
• server: preserve GLiNER word pieces across windows (acbd679)
• server: recheck token links across the scan and warn only for authenticated repeats (#760) (81922aa)
• server: refuse what a chat backend cannot serve in the gateway's words (#823) (fb06db0)
• server: regenerate runtime LoRA model contract (c1af5bf)
• server: register runtime LoRA modules in their bundles (9459c77)
• server: reject unknown numeric LoRA configuration fields (318ef81)
• server: render chat tools as sent on the single-server route (#814) (fabf094)
• server: resolve declared LoRA profile defaults (21ec205)
• server: retain picture text in Docling exports (#754) (39184b5)
• server: roll back a failed LoRA attach and keep adapter sources out of errors (#758) (6d4371c)
• server: send a queued native prompt to a chat-only upstream as one user turn (#871) (b8c557b)
• server: skip the unknown-token id in the ColBERT punctuation skiplist (b078a94)
• server: stop sending the OS user, cwd and pid on /ws/status (#745) (1570745)
• server: support partial QKV LoRA adapters in SGLang (1f686b4)
• server: use cluster cache for decision and entity LoRA (dd20d51)
• server: use full precision for GLiNER PII default profile (b481950)
• server: validate GLiNER2 prompts before document work (#793) (2281ba8)
• server: wake idle workers on admitted work (32c82e5)
• sglang: preserve startup-only LoRA capacity (ae6064c)
• sglang: release LoRA GPU slots on pinned engine unload (ebe6fcc)
• sglang: size GPU memory reservations per model (104ebf1)
• sglang: size memory reservations per model (b342dc5)
• sidecar: expose model-group phase durations (53b2943)
• sidecar: serve long-form audio in its own lane, with per-model batch cost caps (#628) (422b99f)
• systemone: keep refusal codes and surface unreadable results (b1bc9a5)
• systemone: type-check the score ordering and refusal span code (b9f4f76)
• telemetry: preserve model-loading Prometheus metric name (#784) (04f8080)
• whisper: restore SDPA attention after word-timestamp requests (#711) (44b51dc)

Performance Improvements

• examples: speed up Laya router training (b77cb3e)
• models: raise the Hy-MT2-1.8B KV budget to 32768 tokens (a573616)
• server: build GLiNER encoders on the target device (#863) (4ddaaba)