Repository navigation
Highlights
Resources
- Docker Image: intel/llm-scaler-vllm:1.5
- Functionally equivalent to intel/llm-scaler-vllm:0.26.0-b2.
- Documentation: Getting Started and Usage
Ingredients
| Component | Version |
|---|---|
| Container OS | Ubuntu 24.04.4 LTS |
| vLLM baseline | 0.26.0 with Intel XPU patches |
| vllm-xpu-kernels | 0.1.11.2 with Intel XPU patches |
| PyTorch | 2.12.0+xpu |
| Transformers | 5.8.0 |
| Triton XPU | 3.7.1 |
| Intel oneAPI DPC++/C++ Compiler | 2025.3.3 |
| oneCCL installer | 2021.15.9.14 |
| Custom ESIMD kernels | 0.1.0 |
| libze-intel-gpu1 | 26.14.37833.4-1~24.04~ppa1 |
| intel-opencl-icd | 26.14.37833.4-1~24.04~ppa1 |
| intel-ocloc | 26.14.37833.4-1~24.04~ppa1 |
| libigdgmm12 | 22.10.0-1~24.04~ppa1 |
What’s New
-
Software Stack
- Upgrade vLLM from 0.14.0 to the 0.26.0 baseline.
- Upgrade PyTorch from 2.10 to 2.12.
-
Model and Feature Support
- Expand Qwen model coverage, including Qwen3.6-27B, Qwen3.8-27B, and Qwen3.6-35B-A3B.
- Add support for gemma-4-26B-A4B-it and gemma-4-31B-it.
- Support online FP8 and symmetric INT4 quantization.
- Support block-FP8 checkpoints for Qwen3.6-27B, Qwen3.8-27B and Qwen3.6-35B-A3B.
- Support MTP and LoRA serving for Qwen3.6-27B, Qwen3.8-27B, Qwen3.6-35B-A3B, gemma-4-26B-A4B-it, and gemma-4-31B-it.
- Support DFlash for Qwen3.6-27B, Qwen3.8-27B, Qwen3.6-35B-A3B, and Muse-Glimmer-30B.
- Enable FP8 KV cache.
-
Performance Improvements
- Improve Time to First Token (TTFT) for Qwen and Gemma 4 models.
- Improve MTP decoding performance.
- Improve LoRA serving and FP8 KV-cache performance.
-
Correctness and Stability
- Fix prefix caching with MTP.
- Fix incorrect output for sym_int4 models.
- Fix block-FP8 output corruption during concurrent decoding.
- Fix Qwen accuracy and LoRA serving issues.
- Include additional model-loading and runtime bug fixes.
Validated Models
- Qwen3.6-27B
- Qwen3.8-27B
- Qwen3.6-35B-A3B
- gemma-4-26B-A4B-it
- gemma-4-31B-it
- Muse-Glimmer-30B
Known Issues and Limitations
- XPU graph support is experimental.
- Muse-Glimmer multimodal inference requires Transformers 5.15.0;
the image bundles Transformers 5.8.0. - Refer to the release-baseline documentation for model-specific
configuration and limitations.