Skip to content

[NPU] Adapt vLLM-Ascend 0.23 runtime - #9959

Merged
addsubmuldiv merged 3 commits into
modelscope:mainfrom
addsubmuldiv:feat/npu-vllm-0.23
Aug 21, 2026
Merged

[NPU] Adapt vLLM-Ascend 0.23 runtime#9959
addsubmuldiv merged 3 commits into
modelscope:mainfrom
addsubmuldiv:feat/npu-vllm-0.23

Conversation

@addsubmuldiv

@addsubmuldiv addsubmuldiv commented Aug 21, 2026

Copy link
Copy Markdown
Collaborator

PR type

  • Bug Fix
  • New Feature
  • Document Updates
  • More Models or Datasets Support

PR information

Adapt the NPU runtime integration to vLLM-Ascend 0.23.0.

Changes

  • Register the custom OPP path bundled with the vllm-ascend wheel before torch_npu initializes CANN.
  • Detect and use the native vLLM-Ascend fused-MoE LoRA implementation when available.
  • Skip legacy packed-LoRA expansion patches when the installed vLLM version already supports them natively.
  • Update Chinese and English NPU documentation with the vLLM-Ascend 0.23.0 installation instructions:
    • Python 3.12
    • PyTorch 2.10.0 / TorchNPU 2.10.0.post4
    • vLLM 0.23.0 / vLLM-Ascend 0.23.0
    • Required Ascend PyPI mirror indexes

Compatibility

Please install CANN, PyTorch, TorchNPU, and vLLM-Ascend according to the official compatibility matrix.

Experiment results

  • NPU environment initialization
  • vLLM-Ascend inference
  • MoE LoRA rollout/training

@addsubmuldiv addsubmuldiv changed the title Feat/npu vllm 0.23 [NPU] Adapt vLLM-Ascend 0.23 runtime Aug 21, 2026
@addsubmuldiv
addsubmuldiv marked this pull request as ready for review August 21, 2026 04:50
Copilot AI balanced review requested due to automatic review settings August 21, 2026 04:50

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adapts the NPU integration and documentation for vLLM-Ascend 0.23.0.

Changes:

  • Registers wheel-bundled custom OPPs before TorchNPU initialization.
  • Detects native fused-MoE and packed-LoRA capabilities.
  • Updates Chinese and English installation instructions.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

File Description
swift/model/npu_patch/vllm_ascend_lora.py Adds native LoRA capability detection.
swift/model/npu_patch/env.py Bootstraps the custom OPP environment.
docs/source/BestPractices/NPU-support.md Updates Chinese NPU installation guidance.
docs/source_en/BestPractices/NPU-support.md Updates English NPU installation guidance.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines +16 to +20
try:
from vllm_ascend.lora.fused_moe import AscendFusedMoEWithLoRA
except (ImportError, AttributeError):
return False
return AscendFusedMoEWithLoRA is not None
@hjh0119

hjh0119 commented Aug 21, 2026

Copy link
Copy Markdown
Collaborator

Could you confirm whether the earlier vllm_ascend patch can be safely removed starting from version 0.23?

@addsubmuldiv

Copy link
Copy Markdown
Collaborator Author

Could you confirm whether the earlier vllm_ascend patch can be safely removed starting from version 0.23?

For now, could we keep compatibility with vLLM-Ascend 0.18 as well? Removing the legacy patch would require additional validation to ensure there are no regressions for existing NPU users. We can handle the cleanup in a follow-up PR once that verification is complete.

@addsubmuldiv
addsubmuldiv merged commit 86e55a5 into modelscope:main Aug 21, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants