Skip to content

v0.30.0rc1

Pre-release
Pre-release

Choose a tag to compare

@jiangkuaixue123 jiangkuaixue123 released this 19 Oct 08:05
· 75 commits to main since this release
9fcdbd1

Highlights

v0.30.0rc1 upgrades AFD Plugin to vLLM 0.30.0 and completes the transition to ModelRunnerV2 across CUDA and Ascend NPU backends.

  • Runtime upgrade and ModelRunnerV2 migration: Aligns with the vLLM 0.30.0 runtime contract, retires legacy ModelRunnerV1 execution on GPU and NPU, and uses native MRV2 Attention execution with AFD-owned FFN runners.
  • Expanded NPU Async CAM execution: Adds MRV2 Async CAM and layered GMM support, reuses upstream Attention-side DBO for microbatching, and introduces FFN FULL graph execution for supported DeepSeek V4 layered W4A8 configurations on A3/910C while Attention remains eager.
  • Graph and DBO integration: Preserves NPU synchronous CAMP2P eager/graph execution and native two-stage DBO, with fixes for FFN-group padding and stable DSA RoPE metadata during graph replay. CUDA retains supported eager/graph and DBO paths; Qwen3.5/3.6 GDN DBO is temporarily withdrawn.
  • Performance and reliability: Batches Async CAM dispatch prefix writes, resolves combine-send peers lazily, fixes unbounded GPU P2P NCCL receive-buffer cache growth, and corrects NPU layered GMM input validation and Async CAM sequence-parallel configuration.
  • Deployment and documentation: Adds multi-node Kubernetes deployment recipes and a deployment skill, refreshes GPU/NPU runtime and architecture guides, and fixes NPU unit tests for environments without Ascend dependencies.

What's Changed

  • Integrate vLLM 0.30.0 upgrade with current main by @jiaran-king in #425
  • Feat(npu): add support for async cam connector with ModelRunnerV2 by @lirx-pd in #437
  • fix(npu): allow layered GMM with MRV2 by @lirx-pd in #438
  • fix(npu): align MRV2 CAMP2P padding with FFN groups by @lirx-pd in #422
  • Provide recipes and a skill for multi-node deployment on Kubernetes clusters by @shlomitk1 in #421
  • feat(npu): enable native MRV2 eager DBO by @lirx-pd in #441
  • fix(npu): validate layer_index by element count and accept the A8W8 NZ view by @francisleely in #442
  • perf(npu): batch async dispatch send prefix writes by @jiangkuaixue123 in #439
  • perf(npu): resolve async combine send peers lazily by @jiangkuaixue123 in #440
  • Update WeChat community QR-code image by @david6666666 in #445
  • [fix] FFN GPU P2pNcclAFDConnector receive buffer cache grow without bound by @ronenkat in #447
  • fix(npu): disable shared TP for Async CAM model SP by @jiaran-king in #432
  • refactor(npu): reuse upstream DBO for async CAM microbatching by @lirx-pd in #443
  • fix(npu): fix NPU unit tests without Ascend dependencies by @lirx-pd in #460
  • docs: refresh design and GPU/NPU runtime guides by @jiangkuaixue123 in #459
  • [GPU] Migrate AFD execution fully to ModelRunnerV2 by @jiaran-king in #454
  • feat(npu): add Async CAM FFN full graph execution by @jiaran-king in #461
  • [NPU] Retire MRV1 and preserve native MRV2 graph/DBO by @jiaran-king in #462

Full Changelog: v0.26.0rc2...v0.30.0rc1