v3.0.0

What's new in 3.0.0 (2026-07-19)

These are the changes in inference v3.0.0.

New features

• FEAT: Switch default frontend to Next.js app by @OliverBryant in #5111
• feat: (UI) API Key, User Management, refreshToken, change password after first login by @maoyuehui in #5143
• feat(api): validate system message ordering for strict Qwen3-family chat templates by @m199369309 in #5149
• feat(embedding): support BCE embeddings with vLLM by @amumu96 in #5158
• Feat: (UI) Added Security Settings Page、Audit Center Page by @maoyuehui in #5153
• feat: enable advanced database-backed authentication by default by @OliverBryant in #5144
• FEAT: production-ready Docker Compose deployment with offline private PyPI profile by @OliverBryant in #5174
• feat(auth): consolidate to database-backed auth and add offline password reset by @OliverBryant in #5180
• FEAT(webui): show per-model GPU memory in Running Models by @OliverBryant in #5186
• feat(llama.cpp): install GPU xllamacpp wheel from CUDA-matched index by @OliverBryant in #5185
• FEAT: prebuilt pypiserver mirror image as the offline compose package source by @qinrui777 in #5048
• FEAT: slim GPU Docker image without pre-installed inference engines by @OliverBryant in #5188

Enhancements

• ENH: update models JSON [audio] by @XprobeBot in #5142
• ENH: update models JSON [embedding] by @XprobeBot in #5159
• ENH: update 2 models JSON ("CogVideoX-2b", "HunyuanVideo") by @yiboyasss in #5164
• ENH: update model "CogVideoX-5b" JSON by @yiboyasss in #5165
• ENH: update models JSON [audio] by @XprobeBot in #5171
• ENH: update models JSON [embedding, rerank] by @XprobeBot in #5192
• ENH: update models JSON [llm] by @XprobeBot in #5197
• ENH: update models JSON [llm] by @XprobeBot in #5199
• bld: Fix GitHub gpu ci by @OliverBryant in #5107
• bld: Narrow GPU CI triggers and drop always-skipped tests by @OliverBryant in #5182

Bug fixes

• fix(auth): advanced-auth live-read via DB-current permissions by @m199369309 in #5135
• fix(vllm): flashinfer AOT post-install hook for sm_120 Blackwell by @m199369309 in #5137
• fix(audio): bump funasr pin for Fun-ASR-Nano models to ~=1.3 by @m199369309 in #5140
• fix(frontend): route params via usePathname(); remove Gradio, add LaTeX rendering by @OliverBryant in #5146
• fix: replace Korean text with Japanese in ja locale launchModel keys by @kejhz653-stack in #5154
• fix(embedding): implement truncate_prompt_tokens, OOM eviction, and wait_for_load after recovery by @m199369309 in #5151
• fix(tool_parser): fix deepsee-v4 stream tool calls error by @llyycchhee in #5155
• fix(worker): use os.replace for atomic launch_args persist on Windows (#5161) by @Anai-Guo in #5162
• fix(supervisor): resolve comma-separated worker_ip in sharded launch (#5138) by @Anai-Guo in #5160
• fix(llama.cpp): resolve mmproj against the model directory so image input works by @Anai-Guo in #5170
• fix(oauth2): revoke refresh tokens on password reset by @OliverBryant in #5177
• Fix: (UI) copyToClipboard 、fix the locales Running Model Page by @maoyuehui in #5175
• fix(core): free GPU memory when stopping pytorch rerank models by @OliverBryant in #5173
• BUG: sglang venv inherits incompatible parent transformers under skip_installed by @OliverBryant in #5181
• fix: honor virtualenv exemption in engine matching for multimodal LLM, embedding and rerank by @OliverBryant in #5179
• fix(image/ocr): import torchvision lazily in deepseek_ocr (#5189) by @Anai-Guo in #5191
• fix(worker): create model subpool after virtualenv is installed by @kejhz653-stack in #5196
• fix: pass torch version to offline package builds by @qinxuye in #5200
• fix: mirror dependencies declared by wheels built from sdists by @OliverBryant in #5202

Documentation

• doc: update v2.12.0 release docs by @qinxuye in #5136
• doc: xinference release notes links by @OliverBryant in #5139
• DOC: document Xinference 3.0 migration and breaking changes by @OliverBryant in #5194

Others

• ci: narrow GPU test group detection to paths that actually affect GPU tests by @OliverBryant in #5148
• refactor: move XINFERENCE_MODEL_ACTOR_AUTO_RECOVER_LIMIT to constants.py by @m199369309 in #5152
• ci: fix Metal CI failures with latest MLX on macOS 26 runners by @OliverBryant in #5125
• docs(monitor): enhance metrics documentation with cluster label, status detail, and HTTP middleware by @m199369309 in #5184
• chore: remove legacy React web UI now that Next.js frontend is the default by @OliverBryant in #5145

Full Changelog: v2.12.0...v3.0.0