v2.3.1

GPUStack v2.3.1

🚀 Model Catalog Updates

LLM: Kimi-K3.1-Pro, Mistral-Large-4-1T-A52B, Nanbeige4.3-33B.

✨ New Features & Enhancements

• Decision Model Serving: Deploy decision models through vLLM or llama.cpp and access them through /v1/systemone or the Decision Playground. Supports choice, score, and boolean answers for model selection, classification, and lightweight judging tasks (#6436).
• Support CPU-offloaded lookup-table resource estimation for supported vLLM and SGLang configurations, including PLE and Engram. Deduct offloaded table weights from estimated VRAM and reserve RAM on participating workers. RAM and VRAM requirements can be overridden independently (#6417, #6492).
• Remove base-model metadata filtering from LoRA adapter selection, allowing users to select adapters whose metadata does not exactly match the deployment model (#6486).
• Support fractional request rates below one request per second, including the minimum request rate used for automatic searches (#6410).
• Improve SLO search and primary-result selection: report the highest measured load satisfying all configured latency targets and the request-success requirement, track the total-token-throughput peak separately, and leave the primary result empty when no point passes. Continue searching beyond throughput plateaus and report protective stops and unresolved boundaries accurately (#6612).
• Support exact label filtering in worker lists (#6521).
• Add a standalone Community Backend Marketplace button to improve discoverability (#6489).

🔒 Security Fixes

• Fixed unauthorized use of the server-configured Hugging Face token through the model proxy by restricting permitted proxy operations and enforcing authorization for token use (GHSA-4g54-8xx4-7gfw).
• Fixed server-side code execution during model configuration evaluation by reading configuration JSON without executing Python supplied by model repositories (GHSA-gjf9-3p3r-9h9c).
• Strengthen cluster management and registration-token authorization by restricting system credentials to permitted runtime operations and their assigned cluster resources (GHSA-9vwx-hfm8-g59g).
• Enforce immutable resource ownership during updates to prevent changes that bypass resource access controls (GHSA-33f6-vcmh-2xfm).
• Strengthen Kubernetes API proxy authorization by requiring cluster management permissions for operations beyond read-only access to permitted workloads (GHSA-mv55-58pg-4xgf).
• Fixed Hugging Face token and container registry password disclosure in cluster responses by masking credentials in public responses and watch events (GHSA-7345-vjfr-2hcx).
• Strengthen cloud credential authorization when creating or updating a cluster, validating credential access and ownership before binding and provisioning (GHSA-ff8q-w4m5-4h73).
• Strengthen model instance creation authorization by validating access to the parent model and associated resources before creating an instance (GHSA-mp49-qvg2-jp9f).
• Fixed unauthorized updates to platform-wide metrics configuration by requiring platform administrator permissions (GHSA-6c24-hv2c-xgp4).
• Strengthen dashboard data access controls so selecting a cluster preserves the caller's permissions for viewing model activity and token usage (GHSA-h4h6-j8jq-7mxm).
• Fixed unauthenticated access through the forward proxy's /metrics path. Every relayed request now requires authentication, including metrics scraping through worker tunnels (#6629).

🐛 Bug Fixes

• Fixed SGLang distributed PD deployment startup failures caused by port conflicts by separating bootstrap and rendezvous ports and reserving all required data-parallel attention (DPA) ports on participating workers (#6749).
• Fixed distributed vLLM deployments failing to recover when main and subordinate nodes restarted independently and repeatedly interrupted each other's startup (#6669).
• Fixed Gemma 4 model configuration parsing, including heterogeneous attention settings that prevented scheduling (#6561).
• Fixed GGUF VRAM overestimation caused by using the model's training context length instead of the configured context length (#6611).
• Fixed extended KV cache configuration changes not triggering resource re-evaluation (#6662).
• Fixed PD Router file-descriptor exhaustion under high request loads by increasing the soft limit for routers using the bundled runner image (#6607).
• Fixed invalid deployment names causing HTTP 500 when automatic routing is enabled. Names are validated before model and route creation (#6579).
• Fixed inference backend usage-check failures being treated as “not in use”, preventing unsafe changes when usage cannot be determined (#6559).
• Fixed non-streaming inference requests converting backend 4xx responses into HTTP 503, preserving the backend's status code and error body (#6660).
• Fixed custom OpenAI provider URL prefixes being dropped on unmapped requests, including /v1/messages/count_tokens (#6438).
• Fixed Doubao provider model discovery returning an empty list by retaining models whose lifecycle status still permits serving (#6687).
• Fixed stale ready-target counts in model route watch updates, allowing route readiness to update without refreshing the page (#6619).
• Fixed GPU instances remaining in Deleting indefinitely when their cluster is unreachable, adding an explanatory status message and a configurable wait limit (#6552).
• Fixed Ascend devices being reported as unhealthy because of historical ECC records, through the bundled Operator update (#6594).
• Fixed worker client connections remaining in CLOSE-WAIT after short-lived clients and watch connections were no longer used (#6531).
• Fixed cluster-specific image settings being overridden by the server image configuration (8dd6aed2).
• Fixed previous-run logs missing subordinate-node output after distributed deployment recovery (#6661).
• Fixed misaligned GPU and VRAM information in the worker list (#6634).
• Fixed benchmark snapshots showing another launch's backend parameters. Reports use verified instance startup snapshots when available and identify fallback configuration as reference-only (#6615).
• Fixed benchmark failures after renaming a deployed model, preserving the model name used by the running instance (#6589).
• Fixed incomplete statistics for completed benchmark stages while later stages are still running (#6613).
• Fixed benchmark processor arguments being discarded, which caused failures with custom tokenizers such as Kimi-K3 (#6588).

🔧 Built-in Inference Backend Updates

The following versions are newly bundled with v2.3.1. Additional backend versions can be delivered through OTA updates.

New Arrival

• CUDA 13.0
  • vLLM 0.33.0
  • SGLang 0.5.23
• ROCm 7.2
  • vLLM 0.33.0
  • SGLang 0.5.23
• ROCm 10.0
  • vLLM 0.33.0
• CANN 9.1
  • SGLang 0.5.23 / 0.5.22 (950 / A3)