Skip to content

Support vLLM v0.28 accelerator memory APIs - #104

Merged
yeahdongcn merged 5 commits into
MooreThreads:mainfrom
Joey-gvwal:vllm-v028-accelerator-memory-info
Aug 20, 2026
Merged

Support vLLM v0.28 accelerator memory APIs#104
yeahdongcn merged 5 commits into
MooreThreads:mainfrom
Joey-gvwal:vllm-v028-accelerator-memory-info

Conversation

@Joey-gvwal

@Joey-gvwal Joey-gvwal commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Route torch.accelerator.get_memory_info() to torch.musa.mem_get_info() only on torch_musa versions before 2.11.0.post2. The version check strips the local +musa... suffix so post2 and later keep the fixed unified accelerator implementation; missing or malformed versions conservatively retain the workaround.
  • Exclude torchada-owned stable compatibility headers from in-place CUDA-to-MUSA rewriting during editable/source builds.
  • Bump torchada to 0.1.83 across package metadata, runtime version, tests, benchmark history, and documentation.

Validation

S5000 / torch_musa 2.11 post1

  • Candidate: 4357d66fb3f4e8128ae800cf28740294c65a69f3
  • Image: sh-harbor.mthreads.com/mcctest/vllm:v0.24.0-ph1-5.2.0-torch2.11.0.post1-20260820
  • Image digest: sha256:c9b7268be71cbba113696006fd84b139d7a3a0f7d1ac21b141b1631e2831e67a
  • Hardware: one MTT S5000, driver 5.2.0-server
  • Runtime: Python 3.10.12, torch and torch_musa 2.11.0.post1+musa5.2.0, torchada 0.1.83
  • Baseline/candidate A/B: the unpatched post1 torch.accelerator.get_memory_info() reproduced NotImplementedError: getMemoryInfo is not implemented for this allocator yet; the PR candidate passed and resolved it to torch.musa.mem_get_info.
  • Runtime inventory confirmed nine post1 memory overrides; empty_host_cache was absent from both modules. A real 16 MiB MUSA allocation was reflected by memory_allocated, memory_stats, peak/reset APIs, empty_cache, synchronize, and get_memory_info.
  • get_memory_info passed with None, integer, string, and torch.device arguments. Same-name and remapped fallback behavior also passed when the official accelerator API was absent.
  • Focused accelerator tests: 40 passed with no skips.
  • Other changed-surface tests (test_inplace_porting.py, test_platform.py): 26 passed.
  • Full torchada suite: 465 passed, 19 skipped, 11 warnings in 55.43s.

Other validation

  • Post1/post2 version-boundary, local-suffix trimming, malformed-version fallback, post2 native-selection, and post2 missing-API fallback tests passed. The post2 selection checks are simulated until an actual 2.11.0.post2 wheel is available.
  • Pre-commit hooks passed for all files changed by the review fix.
  • Existing PR validation: vLLM-MUSA v0.28.0 source build/install with editable torchada on an 8-GPU MUSA 5.2.0 environment.

Scope note: this compatibility layer wraps the top-level torch.accelerator.* API used by vLLM. The separate torch.accelerator.memory.* submodule is outside this PR's scope.

@yeahdongcn

Copy link
Copy Markdown
Collaborator

@lijing-mt Please be aware of this issue.

Comment thread src/torchada/_patch.py Outdated
@yeahdongcn yeahdongcn changed the title support Vllm v028 Support vLLM v0.28 accelerator memory APIs Aug 20, 2026
@yeahdongcn

Copy link
Copy Markdown
Collaborator

S5000 merge-readiness validation completed at head 4357d66 on the requested torch2.11.0.post1 image (digest sha256:c9b7268...e67a). The unpatched baseline reproduced the allocator NotImplementedError; the candidate passed the top-level override/remap/fallback A/B, allocation-backed memory checks, 40 focused accelerator tests, 26 other changed-surface tests, and the full suite (465 passed, 19 skipped). The PR body now contains the exact environment and scope.

@yeahdongcn
yeahdongcn merged commit 6cf1c3f into MooreThreads:main Aug 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants