Skip to content

llm-scaler-vllm beta release 0.14.0-b8

Pre-release
Pre-release

Choose a tag to compare

View all tags
@liu-shaojun liu-shaojun released this 02 Mar 07:12
vllm-0.14.0-b8
4a65586
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Highlights

Resources

Ingredients

Ingredients Version
Host OS Ubuntu 24.04 Desktop/Server
vllm 0.14.0
PyTorch 2.10
OneAPI 2025.3.2
OneCCL 15.7.8
UMD Driver 25.40.36300.8
KMD Driver 6.17.0-1007.7
GuC Firmware 70.55.3
XPU Manager 1.3.5
Offline Installer 26.13.7.1

What’s new

  • vLLM:

    • Upgrade: vLLM upgrade to 0.14.0, Pytorch upgrade to 2.10. oneAPI uplifted to 2025.3.2(hotfix) with LTS support on UR adaptor v2. Oneccl upgrade to 2021.15.7.8.
    • Int4 onednn optimizations are included and up to 25% throughput improvement is achieved VS last release.
    • Bug fixes.
    • Added support for Qwen3-VL-Reranker-2B/8B
    • Added support for Qwen3-VL-Embedding-2B/8B
    • Added support for GLM-4.7-Flash
    • Added support for Ministral models
    • Added support for DeepSeek-OCR-2
    • Added support for Qwen3-Coder-Next
    • Fix InternVL issue

  • Offline Installer:

    • Add 24.04 support
    • Add script to configure PCIe switch downstream port ACS
    • Add script for P2P issue diagnostic
Assets 2
Loading