v0.30.0rc1

Highlights

v0.30.0rc1 upgrades AFD Plugin to vLLM 0.30.0 and completes the transition to ModelRunnerV2 across CUDA and Ascend NPU backends.

• Runtime upgrade and ModelRunnerV2 migration: Aligns with the vLLM 0.30.0 runtime contract, retires legacy ModelRunnerV1 execution on GPU and NPU, and uses native MRV2 Attention execution with AFD-owned FFN runners.
• Expanded NPU Async CAM execution: Adds MRV2 Async CAM and layered GMM support, reuses upstream Attention-side DBO for microbatching, and introduces FFN FULL graph execution for supported DeepSeek V4 layered W4A8 configurations on A3/910C while Attention remains eager.
• Graph and DBO integration: Preserves NPU synchronous CAMP2P eager/graph execution and native two-stage DBO, with fixes for FFN-group padding and stable DSA RoPE metadata during graph replay. CUDA retains supported eager/graph and DBO paths; Qwen3.5/3.6 GDN DBO is temporarily withdrawn.
• Performance and reliability: Batches Async CAM dispatch prefix writes, resolves combine-send peers lazily, fixes unbounded GPU P2P NCCL receive-buffer cache growth, and corrects NPU layered GMM input validation and Async CAM sequence-parallel configuration.
• Deployment and documentation: Adds multi-node Kubernetes deployment recipes and a deployment skill, refreshes GPU/NPU runtime and architecture guides, and fixes NPU unit tests for environments without Ascend dependencies.

What's Changed

• Integrate vLLM 0.30.0 upgrade with current main by @jiaran-king in #425
• Feat(npu): add support for async cam connector with ModelRunnerV2 by @lirx-pd in #437
• fix(npu): allow layered GMM with MRV2 by @lirx-pd in #438
• fix(npu): align MRV2 CAMP2P padding with FFN groups by @lirx-pd in #422
• Provide recipes and a skill for multi-node deployment on Kubernetes clusters by @shlomitk1 in #421
• feat(npu): enable native MRV2 eager DBO by @lirx-pd in #441
• fix(npu): validate layer_index by element count and accept the A8W8 NZ view by @francisleely in #442
• perf(npu): batch async dispatch send prefix writes by @jiangkuaixue123 in #439
• perf(npu): resolve async combine send peers lazily by @jiangkuaixue123 in #440
• Update WeChat community QR-code image by @david6666666 in #445
• [fix] FFN GPU P2pNcclAFDConnector receive buffer cache grow without bound by @ronenkat in #447
• fix(npu): disable shared TP for Async CAM model SP by @jiaran-king in #432
• refactor(npu): reuse upstream DBO for async CAM microbatching by @lirx-pd in #443
• fix(npu): fix NPU unit tests without Ascend dependencies by @lirx-pd in #460
• docs: refresh design and GPU/NPU runtime guides by @jiangkuaixue123 in #459
• [GPU] Migrate AFD execution fully to ModelRunnerV2 by @jiaran-king in #454
• feat(npu): add Async CAM FFN full graph execution by @jiaran-king in #461
• [NPU] Retire MRV1 and preserve native MRV2 graph/DBO by @jiaran-king in #462

Full Changelog: v0.26.0rc2...v0.30.0rc1

v0.26.0rc2

What's Changed

• Update WeChat community QR-code image by @david6666666 in #262
• Add AFD support for CUDA GPU ModelRunnerV2 by @jiaran-king in #245
• add MRV2 for NPU without DBO by @lirx-pd in #257
• [Bugfix] fix bug in unit tests by @lirx-pd in #266
• [CI] switch DeepSeek-V2-Lite E2E from h100_4 to l4_4 by @yenuo26 in #250
• [Enhancement] Strengthen pre-commit hooks and replace CPU-only CI by @yenuo26 in #271
• Add CANN terms to typos allowlist by @jiangkuaixue123 in #277
• docs: refresh module design documentation by @jiangkuaixue123 in #280
• fix(e2e): clean up NPU async FFN process trees by @ShwStone in #279
• Bugfix: align NPU MRV1 signatures with vLLM-Ascend commit 80d8c194f by @lirx-pd in #282
• fix: gate GPU FFN graph replay on Attention run mode by @Betelgeu in #270
• [CI] add Buildkite CUDA weekly pipeline by @yenuo26 in #268
• Update WeChat community QR-code image by @david6666666 in #288
• Add connector-scoped HCCL buffer configuration by @ShwStone in #291
• fix: restore request-count DPLB for async Attention DP by @ShwStone in #289
• test: align coordinator warning assertion by @ShwStone in #298
• [CI] Move L4 Buildkite jobs onto k8s GPU pool by @yenuo26 in #299
• [feat][model]v0.26.0 camasync dsv4 by @bjf-frz in #300
• [NPU] Port DSV4 CAM fixes to main by @jiangkuaixue123 in #311
• Move .dockerignore to the repo root so it is actually read by @ronenkat in #314
• Update run-e2e skill fixing inconsistencies for the Qwen suite by @ronenkat in #306
• Update WeChat community QR-code image by @david6666666 in #316
• test: fix Async CAM weekly CI regressions by @jiangkuaixue123 in #319
• fix(npu): bind FFN worker CPUs during startup by @jiangkuaixue123 in #321
• [PERF][NPU] optimize DSV4 CAM FFN control path by @bjf-frz in #320
• test: guard NPU CPU-binding tests before runtime imports by @jiangkuaixue123 in #323
• [Docs] Add project logo and presentation link by @david6666666 in #326
• docs: add review-pr agent skill by @hsliuustc0106 in #329
• fix(gpu): reuse the AFD world's store for subgroup rendezvous by @swjeong9 in #328
• [Feature] Relax the divisibility constraint in P2pNcclAFDConnector by @swjeong9 in #309
• Update WeChat community QR-code image by @david6666666 in #339
• fix(npu): support W4A8 expert weights in CAM MLP by @bjf-frz in #342
• [NPU] add A5/ascend950 support for A2E and E2A ops by @wangyibo1005 in #324
• test(npu): fix MoE quantization contract assertion after W4A8 support by @bjf-frz in #347
• test(npu): add DSV4 Flash async CAM concurrent E2E by @jiangkuaixue123 in #352
• test(npu): expect the NPU DSV4 registration on Ascend hosts by @ksiyuan in #350
• feat(npu): DeepSeek V4 over the synchronous CAMP2P boundary by @ksiyuan in #340
• [NPU] Restructure csrc/npu around the CANN npu_op_* build system by @francisleely in #354
• fix: resolve pre-commit failures in Ascend build tests by @jiangkuaixue123 in #358
• [bug] Declare and install e2e test time deps in pyproject and image by @ronenkat in #357
• Update WeChat community QR-code image by @david6666666 in #362
• Add routed-only asynchronous CAM operators to the AFD NPU build by @GuangyuZhu04 in #361
• [NPU] Migrate async CAM to source-built routed-only operators by @jiangkuaixue123 in #363
• [NPU] Revert DSV4 attention router FP32 linear by @jiangkuaixue123 in #366
• [NPU] Ignore generated Ascend runtime artifacts by @jiangkuaixue123 in #367
• Add grouped_matmul_swiglu_quant_v2_layered CAM operator by @simonkorl in #365
• [NPU] Fuse DSV4 Async CAM shared expert activation and quantization by @jiangkuaixue123 in #371
• [NPU] Fix layered GMM kernel configuration lookup by @jiangkuaixue123 in #373
• [Test][NPU] Add HCCL precision and performance harnesses for routed async operators by @GuangyuZhu04 in #374
• Vendor the layered grouped_matmul_layered CAM operator by @francisleely in #376
• fix(npu): define tensor list type for layered GMM binding by @jiangkuaixue123 in #378
• [BugFix][NPU] Bind DBO microbatch threads to their target device by @lirx-pd in #383
• fix(npu): accept device group lists in layered GMM by @jiangkuaixue123 in #384
• bugfix: DBO E2E tests can pass without splitting live requests into ubatches by @yujuancao07 in #351
• feat(npu): add DeepSeek V4 async CAM W4A8 layered FFN path by @jiangkuaixue123 in #387
• fix: Treat zombie-only process groups as terminated in E2E teardown by @ronenkat in #393
• Make afd_process_group timeout configurable by @shlomitk1 in #400
• fix(npu): the install problem with CANN9.1 by @lirx-pd in #399
• fix(gpu): warm full-batch FFN workspace before graph capture by @lirx-pd in #404
• [NPU] Use HCCL_BUFFSIZE for CAM host tiling by @GuangyuZhu04 in #410
• perf(npu): reuse dispatch prefixes in async combine recv by @jiangkuaixue123 in #403
• Update WeChat community QR-code image by @david6666666 in #415
• fix(tests): restore Legacy DBO smoke coverage by @jiaran-king in #416
• Update WeChat community QR-code image by @david6666666 in #423
• fix(npu): validate layer_index at the aclnn entry in layered GMM by @francisleely in #427
• test(npu): add DeepSeek V4 sync CAMP2P E2E case by @ksiyuan in #359

New Contributors

• @Betelgeu made their first contribution in #270
• @ronenkat made their first contribution in #314
• @swjeong9 made their first contribution in #328
• @wangyibo1005 made their first contribution in #324
• @ksiyuan made their first contribution in #350
• @francisleely made their first contribution in #354
• @GuangyuZhu04 made their first contribution in #361
• @simonkorl made their first contribution in #365
• @shlomitk1 made their first contribution in #400

Full Changelog: v0.26.0rc1...v0.26.0rc2

v0.26.0rc1

Highlights

v0.26.0rc1 upgrades the AFD Plugin from the vLLM 0.19.1 runtime contract to vLLM 0.26.0 across both CUDA and Ascend NPU backends.

• Runtime upgrade: Aligns GPU and NPU model runners, connectors, CUDA/ACL graph execution, DBO, and compatibility patches with the vLLM and vLLM-Ascend 0.26 APIs.
• Expanded CUDA model support: Adds AFD integrations for Qwen3 MoE, Qwen3.5/3.6 MoE, and DeepSeek V4 while preserving native model construction and weight-loading behavior.
• Improved NPU async execution: Ports token-balanced, two-stage Async CAM MoE ubatching to v0.26, restores correct first-request startup behavior, and adds focused recipes, tests, and troubleshooting guidance.
• Performance and reliability: Overlaps Attention and FFN model loading, fixes CUDA metadata handling for asymmetric topologies and multi-ubatch DBO, and strengthens connector/runtime lifecycle behavior.
• Stronger validation: Introduces Buildkite CUDA unit and DeepSeek-V2-Lite E2E gates and expands GPU/NPU coverage across eager, graph, DBO, topology, and model scenarios.

What's Changed

• Fix pure-text AFD model config identity by @ShwStone in #174
• Update WeChat community image by @david6666666 in #187
• fix(npu): preserve AFD DBO config during auto worker selection by @jiangkuaixue123 in #195
• docs: fix afd_role_rank derivation guidance by @ShwStone in #193
• bugfix:DBO ubatch NPUGraph capture passes per-stage metadata to the standard full-graph updater by @yujuancao07 in #178
• refactor: make AFD role rank runtime-only by @ShwStone in #197
• [v0.26.0] Upgrade GPU and NPU backends by @jiangkuaixue123 in #186
• refactor(gpu): remove P2P ordering token by @jiaran-king in #204
• test(npu): validate runtime class resolution order by @ShwStone in #212
• test: fix stale role-rank and forward-context fixtures by @sinpeyw in #214
• fix(npu): skip empty Async CAM FFN work by @ShwStone in #213
• [Bugfix] Avoid cross-ubatch Attention metadata reuse in AFD DBO by @sinpeyw in #217
• [Feature] Add repository-backed model adaptation skill by @jiaran-king in #205
• Update WeChat community QR-code image by @david6666666 in #220
• [Bugfix] Fix CUDA FFN token metadata for asymmetric topologies by @sinpeyw in #219
• refactor e2e tests by @yujuancao07 in #222
• feat(npu): port token-balanced Async CAM MoE ubatching to vLLM 0.26 by @ShwStone in #203
• feat: add NPU runtime upgrade skill by @jiangkuaixue123 in #231
• [Feature] Add Qwen3 MoE CUDA AFD support by @sinpeyw in #216
• feat: add GPU runtime upgrade skill by @jiaran-king in #235
• Refine test cases by @yujuancao07 in #238
• [CI] add Buildkite CUDA ready/merge pipeline (unit + DeepSeek-V2-Lite E2E) by @yenuo26 in #221
• Add missing skill UI metadata by @jiangkuaixue123 in #246
• [Feature] Add deterministic forced routing for CUDA MoE benchmarks by @sinpeyw in #243
• feat: Parallelize Attention and FFN model loading across AFD connectors by @ShwStone in #244
• [Bugfix]Fix async CAM first-request deadlock by @jiangkuaixue123 in #248
• test(e2e): run DeepSeek-V2-Lite baseline with DP2 EP2 by @yujuancao07 in #249
• Add Qwen3.6 MoE CUDA AFD implementation support by @shanyulu in #181
• test(e2e): add Qwen3 MoE CUDA coverage by @sinpeyw in #241
• test(e2e): run DeepSeek-V2-Lite baseline with DP4 EP4 by @yujuancao07 in #252
• NPU Async Connector: tests and docs by @ShwStone in #236
• fix(recipe): correct DeepSeek-V2-Lite proxy path by @like955 in #253
• V0.26.0 support for dsv4 (gpu) by @lirx-pd in #191
• test(e2e): add Qwen3.6 MoE CUDA coverage by @shanyulu in #256
• Update WeChat QR code by @jiangkuaixue123 in #258
• Update WeChat community QR-code image by @david6666666 in #259

New Contributors

• @yenuo26 made their first contribution in #221
• @shanyulu made their first contribution in #181
• @like955 made their first contribution in #253
• @lirx-pd made their first contribution in #191

Full Changelog: v0.19.1rc1...v0.26.0rc1

v0.19.1rc1

What's Changed

• Add GitHub issue and PR templates by @jiangkuaixue123 in #2
• Migrate AFD into external vLLM plugin by @jiangkuaixue123 in #1
• Add AFD docs and DeepSeekV2-Lite examples by @jiangkuaixue123 in #5
• Add Ascend NPU decode AFD runtime by @jiangkuaixue123 in #8
• [codex] Clean up runtime import boundaries by @jiangkuaixue123 in #13
• [codex] Add Ascend NPU AFD runtime support by @jiangkuaixue123 in #14
• [codex] Add NPU AFD runner profilers by @jiangkuaixue123 in #16
• [codex] Fix NPU AFD DBO ubatch overlap by @jiangkuaixue123 in #17
• Fix NPU FFN token aggregation graph key by @jiangkuaixue123 in #18
• Load separate weights for A-side and F-side workers by @yujuancao07 in #15
• [codex] Remove afd_size from AFD config by @jiangkuaixue123 in #19
• Add GPU profiler support by @jiangkuaixue123 in #20
• Align connector kwargs interface by @jiangkuaixue123 in #23
• [Feature]: Support Tensor Parallelism for AFD by @yujuancao07 in #22
• bugfix: DeepEP Mode Crashes with AttributeError: 'ForwardContext' object has no attribute 'afd_metadata' by @yujuancao07 in #25
• [Feature]: Add GPU/NPU AFD E2E coverage for graph modes, DBO, topology ratios, and precision by @yujuancao07 in #28
• ds-v2-lite(GPU) recipe by @yujuancao07 in #30
• Add NPU profiler stack toggle by @jiangkuaixue123 in #38
• [Doc] Refresh AFD plugin docs by @jiangkuaixue123 in #43
• [codex] Split csrc native sources by backend by @jiangkuaixue123 in #44
• [chore]: Rename prefill_decode_multiplexing to prefill_decode_colocated, update Readme by @yujuancao07 in #34
• Refine runtime import boundaries by @jiangkuaixue123 in #46
• Remove migration-specific code comments and phase terminology by @jiaran-king in #45
• Clarify AFD config fields and cleanup helpers by @jiangkuaixue123 in #47
• Clean up redundant helper code by @jiangkuaixue123 in #50
• [chore]: renaming prefill_decode_colocated to prefill_decode_colocation by @specture724 in #51
• [Bugfix] Skip Ascend ops by default on GPU builds by @jiangkuaixue123 in #57
• [codex] Simplify CAMP2P send attention output by @jiangkuaixue123 in #60
• [codex] Unify AFD connector metadata contracts by @jiangkuaixue123 in #61
• Sync fork main validation changes by @jiangkuaixue123 in #62
• [codex] Document connector contracts and simplify Ascend FFN runner by @jiangkuaixue123 in #63
• narrow obvious Any types by @jiaran-king in #59
• [codex] Use tensor fields for AFD DP metadata by @jiangkuaixue123 in #66
• [Feature] Add NPU async DP framework and Async Cam connector by @jiangkuaixue123 in #64
• Fix GPU P2P metadata for 2A1F by @jiaran-king in #69
• [chore]: move benchmark.py and decode_bench.py to tools/benchmarks/ by @specture724 in #52
• support force load balance by @yujuancao07 in #48
• [codex] Fix Ruff CI failures by @jiangkuaixue123 in #71
• [codex] Handle empty async CAM FFN ranks by @jiangkuaixue123 in #72
• [codex] Add NPU binary packages and dataset by @jiangkuaixue123 in #74
• [Fix]: add proxy-based PD disaggregation for 2P1A1F by @specture724 in #55
• [codex] Remove dataset JSONL by @jiangkuaixue123 in #76
• Remove unused async CAM extra config by @jiangkuaixue123 in #78
• [Docs]: add AGENTS development guidelines by @jiangkuaixue123 in #80
• [Fix] NPU cudagraph compile failure from runtime UnionType by @specture724 in #82
• [refactor]: move dp metadata codec logic out of p2p, reuse it in camp2p by @specture724 in #79
• [feat]: pre-commit hook added for format and lint by @specture724 in #85
• [Refactor] Refactor async-DP engine patch by @jiangkuaixue123 in #84
• rename AFDRecvOutput to AFDAttnOutput by @jiangkuaixue123 in #92
• [Bugfix] Fix async CAM routed token count by @jiangkuaixue123 in #95
• [Hotfix]: The logic of enabling ubatch should consider the first ubatch is not empty by @specture724 in #100
• Remove unused DP metadata timeout parameter by @jiangkuaixue123 in #102
• [Fix]: p2pconnector support for multi-tpdp by @specture724 in #94
• Refactor Ascend runtime compatibility patches by @jiangkuaixue123 in #98
• Rename AFD connectors by @jiangkuaixue123 in #103
• [recipe] Add CAMAsyncAFDConnector for DeepSeek-V3.2 by @jiangkuaixue123 in #77
• [recipe] Group GPU recipes by connector by @jiangkuaixue123 in #109
• Share agent instructions across tools by @jiangkuaixue123 in #110
• Improve README support overview by @jiangkuaixue123 in #111
• [Fix]: Fake torch_npu load for test_async_cam_connector.py by @specture724 in #112
• Refactor: Renaming classes in metadata scope by @specture724 in #108
• Add async CAM DeepSeekV2-Lite e2e smoke by @jiangkuaixue123 in #117
• Fix vLLM 0.19.1 CPU test failures by @jiangkuaixue123 in #118
• docs: streamline README status and E2E guidance by @jiangkuaixue123 in #122
• docs: update AFD architecture diagram by @jiangkuaixue123 in #116
• [Feat]: DeepSeek-V3.2 Synchronous Decode with CAMP2pAFDConnector on Ascend NPU by @yujuancao07 in #67
• docs: correct connector recommend stages by @jiangkuaixue123 in #124
• docs: organize recipes by connector and model by @jiangkuaixue123 in #125
• [Refactor]: Remove metadata helpers by @specture724 in #119
• [Doc]: Docs and doc strings added for nccl p2p connector by @specture724 in #123
• docs: refresh runtime documentation by @jiangkuaixue123 in #126
• [Doc] Add CAM async connector usage guide by @jiangkuaixue123 in #127
• Add repository code owners by @hsliuustc0106 in #130
• Remove obsolete multistream configuration by @jiangkuaixue123 in #132
• Add documentation for the synchronous Ascend NPU CAMP2pAFDConnector by @yujuancao07 in #131
• Implement RFC89 AFD config cleanup by @jiaran-king in #96
• docs: add Ascend NPU installation guide by @jiaran-king in #135
• Rename Ascend module directories to NPU by @jiangkuaixue123 in #134
• docs: fix NPU verification import after module rename by @jiaran-king in #136
• Remove unused environment variables by @jiangkuaixue123 in #137
• Refactor/split control plane by @specture724 in #128
• [misc]: remove stepmesh by @specture724 in #138
• Document CAM async deployment constraints by @jiangkuaixue123 in ht...