Repository navigation
v0.30.0rc1
Pre-release
Pre-release
·
75 commits
to main
since this release
Highlights
v0.30.0rc1 upgrades AFD Plugin to vLLM 0.30.0 and completes the transition to ModelRunnerV2 across CUDA and Ascend NPU backends.
- Runtime upgrade and ModelRunnerV2 migration: Aligns with the vLLM 0.30.0 runtime contract, retires legacy ModelRunnerV1 execution on GPU and NPU, and uses native MRV2 Attention execution with AFD-owned FFN runners.
- Expanded NPU Async CAM execution: Adds MRV2 Async CAM and layered GMM support, reuses upstream Attention-side DBO for microbatching, and introduces FFN FULL graph execution for supported DeepSeek V4 layered W4A8 configurations on A3/910C while Attention remains eager.
- Graph and DBO integration: Preserves NPU synchronous CAMP2P eager/graph execution and native two-stage DBO, with fixes for FFN-group padding and stable DSA RoPE metadata during graph replay. CUDA retains supported eager/graph and DBO paths; Qwen3.5/3.6 GDN DBO is temporarily withdrawn.
- Performance and reliability: Batches Async CAM dispatch prefix writes, resolves combine-send peers lazily, fixes unbounded GPU P2P NCCL receive-buffer cache growth, and corrects NPU layered GMM input validation and Async CAM sequence-parallel configuration.
- Deployment and documentation: Adds multi-node Kubernetes deployment recipes and a deployment skill, refreshes GPU/NPU runtime and architecture guides, and fixes NPU unit tests for environments without Ascend dependencies.
What's Changed
- Integrate vLLM 0.30.0 upgrade with current main by @jiaran-king in #425
- Feat(npu): add support for async cam connector with ModelRunnerV2 by @lirx-pd in #437
- fix(npu): allow layered GMM with MRV2 by @lirx-pd in #438
- fix(npu): align MRV2 CAMP2P padding with FFN groups by @lirx-pd in #422
- Provide recipes and a skill for multi-node deployment on Kubernetes clusters by @shlomitk1 in #421
- feat(npu): enable native MRV2 eager DBO by @lirx-pd in #441
- fix(npu): validate layer_index by element count and accept the A8W8 NZ view by @francisleely in #442
- perf(npu): batch async dispatch send prefix writes by @jiangkuaixue123 in #439
- perf(npu): resolve async combine send peers lazily by @jiangkuaixue123 in #440
- Update WeChat community QR-code image by @david6666666 in #445
- [fix] FFN GPU P2pNcclAFDConnector receive buffer cache grow without bound by @ronenkat in #447
- fix(npu): disable shared TP for Async CAM model SP by @jiaran-king in #432
- refactor(npu): reuse upstream DBO for async CAM microbatching by @lirx-pd in #443
- fix(npu): fix NPU unit tests without Ascend dependencies by @lirx-pd in #460
- docs: refresh design and GPU/NPU runtime guides by @jiangkuaixue123 in #459
- [GPU] Migrate AFD execution fully to ModelRunnerV2 by @jiaran-king in #454
- feat(npu): add Async CAM FFN full graph execution by @jiaran-king in #461
- [NPU] Retire MRV1 and preserve native MRV2 graph/DBO by @jiaran-king in #462
Full Changelog: v0.26.0rc2...v0.30.0rc1