Skip to content

OpenVINO Model Server 2026.3.0

Choose a tag to compare

View all tags
@dtrawins dtrawins released this 04 Aug 22:48
· 9 commits to releases/2026/3 since this release
v2026.3
f4a2934
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

New features

Improved performance, accuracy, memory consumption, and stability, with particular focus on new models featuring linear attention architectures like qwen3.6.

Added new tool parsers for LFM2.5 and MiniCPM-V5 models. They can be used in agentic scenarios with native function calling.

Added support for the Jinja chat template engine as an alternative to Minja for Vision Language Models (VLMs). By default, ovms package with python integration use Jinja while the package without python can use Minja. Jinja provides support for wider set of chat templates, but most models are supported by Minja. Current exceptions are lfm2, miniCPMv5 and Qwen3-Coder

Introduced automatic detection of key runtime parameters for generative models, including model task category, target device, and tool parsers, reducing required configuration and minimizing deployment errors. Default target device is determined based on the priority: dGPU if present, iGPU if present, CPU. If there are several discrete GPUs, the one with more free VRAM will be selected.

Simplified deployment of classic models by enabling direct deployment without versioning by pointing ‘--model_path’ to a model file, in addition to a folder with model versions.

Extended OVMS CLI support for easier configuration and deployment of local models, including the ability to add classic models to multi-model config file alongside generative models.

Preview support for Kokoro model in OpenAI API ‘/audio/speech’ endpoint, including multi-language support via the optional eSpeek component.

Extended support for Automatic Speech Recognition (ASR) models beyond the Whisper model family, enabling more generic audio endpoint support for models such as Qwen-ASR.

Preview support for Qwen3-OMNI models. This pipeline exposes chat/completions and /responses API. On input, is accepted text, images and audio. Model can generate text and audio. Both unary and streaming responses are covered.

Enabled /v1 REST version for generative endpoints. It can be used just like v3 which remains for compatiblities reasons with previous version. v1 version makes easier integration with some AI frameworks assuming v1 for OpenAI endpoints.

Added option --verbose_response which makes chat/completions endpoint sending raw model prompts and raw model response. It can be used for debugging purposes or testing. Verbose response is compatible with llama.cpp.

Bug fixes

Addressed known security vulnerabilities

Discontinued

TensorFlow Server API is now discontinued. KServe API is addressing the same use cases but has more capabilities and more efficient communication pattern.

Known issues

  • Gemma4 models are supported only on CPU. GPU execution might be not reliable yet. Resolution is expected soon.
  • Kokoro is supported on CPU only. GPU and NPU execution is to be enabled soon.
  • OMNI models are supported as a preview feature with limitations - MOE models are not supported yet, concurrency is not possible - requests will be queued in processing.
  • NPU execution on LLM has a limit on max prompt parameter 8k tokens. Model will fail to load with higher parameter. It is expected to be fixed soon.
  • NPU plugin for Qwen3-embeddings and Qwen4-reranking fails to load the model when allowed context exceeds 8k tokens. In order to user such models, the workaround is to edit config.json in the model folder and set "max_position_embeddings" to be below 8k. It is expected to be fixed soon.
Assets 18
Loading