Repository navigation
·
155 commits
to main
since this release
Highlights
v0.26.0rc1 upgrades the AFD Plugin from the vLLM 0.19.1 runtime contract to vLLM 0.26.0 across both CUDA and Ascend NPU backends.
- Runtime upgrade: Aligns GPU and NPU model runners, connectors, CUDA/ACL graph execution, DBO, and compatibility patches with the vLLM and vLLM-Ascend 0.26 APIs.
- Expanded CUDA model support: Adds AFD integrations for Qwen3 MoE, Qwen3.5/3.6 MoE, and DeepSeek V4 while preserving native model construction and weight-loading behavior.
- Improved NPU async execution: Ports token-balanced, two-stage Async CAM MoE ubatching to v0.26, restores correct first-request startup behavior, and adds focused recipes, tests, and troubleshooting guidance.
- Performance and reliability: Overlaps Attention and FFN model loading, fixes CUDA metadata handling for asymmetric topologies and multi-ubatch DBO, and strengthens connector/runtime lifecycle behavior.
- Stronger validation: Introduces Buildkite CUDA unit and DeepSeek-V2-Lite E2E gates and expands GPU/NPU coverage across eager, graph, DBO, topology, and model scenarios.
What's Changed
- Fix pure-text AFD model config identity by @ShwStone in #174
- Update WeChat community image by @david6666666 in #187
- fix(npu): preserve AFD DBO config during auto worker selection by @jiangkuaixue123 in #195
- docs: fix afd_role_rank derivation guidance by @ShwStone in #193
- bugfix:DBO ubatch NPUGraph capture passes per-stage metadata to the standard full-graph updater by @yujuancao07 in #178
- refactor: make AFD role rank runtime-only by @ShwStone in #197
- [v0.26.0] Upgrade GPU and NPU backends by @jiangkuaixue123 in #186
- refactor(gpu): remove P2P ordering token by @jiaran-king in #204
- test(npu): validate runtime class resolution order by @ShwStone in #212
- test: fix stale role-rank and forward-context fixtures by @sinpeyw in #214
- fix(npu): skip empty Async CAM FFN work by @ShwStone in #213
- [Bugfix] Avoid cross-ubatch Attention metadata reuse in AFD DBO by @sinpeyw in #217
- [Feature] Add repository-backed model adaptation skill by @jiaran-king in #205
- Update WeChat community QR-code image by @david6666666 in #220
- [Bugfix] Fix CUDA FFN token metadata for asymmetric topologies by @sinpeyw in #219
- refactor e2e tests by @yujuancao07 in #222
- feat(npu): port token-balanced Async CAM MoE ubatching to vLLM 0.26 by @ShwStone in #203
- feat: add NPU runtime upgrade skill by @jiangkuaixue123 in #231
- [Feature] Add Qwen3 MoE CUDA AFD support by @sinpeyw in #216
- feat: add GPU runtime upgrade skill by @jiaran-king in #235
- Refine test cases by @yujuancao07 in #238
- [CI] add Buildkite CUDA ready/merge pipeline (unit + DeepSeek-V2-Lite E2E) by @yenuo26 in #221
- Add missing skill UI metadata by @jiangkuaixue123 in #246
- [Feature] Add deterministic forced routing for CUDA MoE benchmarks by @sinpeyw in #243
- feat: Parallelize Attention and FFN model loading across AFD connectors by @ShwStone in #244
- [Bugfix]Fix async CAM first-request deadlock by @jiangkuaixue123 in #248
- test(e2e): run DeepSeek-V2-Lite baseline with DP2 EP2 by @yujuancao07 in #249
- Add Qwen3.6 MoE CUDA AFD implementation support by @shanyulu in #181
- test(e2e): add Qwen3 MoE CUDA coverage by @sinpeyw in #241
- test(e2e): run DeepSeek-V2-Lite baseline with DP4 EP4 by @yujuancao07 in #252
- NPU Async Connector: tests and docs by @ShwStone in #236
- fix(recipe): correct DeepSeek-V2-Lite proxy path by @like955 in #253
- V0.26.0 support for dsv4 (gpu) by @lirx-pd in #191
- test(e2e): add Qwen3.6 MoE CUDA coverage by @shanyulu in #256
- Update WeChat QR code by @jiangkuaixue123 in #258
- Update WeChat community QR-code image by @david6666666 in #259
New Contributors
- @yenuo26 made their first contribution in #221
- @shanyulu made their first contribution in #181
- @like955 made their first contribution in #253
- @lirx-pd made their first contribution in #191
Full Changelog: v0.19.1rc1...v0.26.0rc1