Conversation
HuiyingLi
force-pushed
the
huiyingl/perf-qwen3_5_moe-simplify-state-dict-adapter
branch
from
ebf61f8 to
45e0e40
Compare
HuiyingLi
requested review from
ZhiyuLi-Nvidia,
adil-a,
akoumpa and
hemildesai
as code owners
Contributor
Author
|
/ok to test 45e0e40 |
Same fix as PR #1570 for Qwen3-VL-MoE, applied to Qwen3.5-MoE: - to_hf: removed all_gather_object, full_tensor(), per-expert CPU copies, .to(dtype) casts. Now just renames keys + transpose(1,2). No comms. - from_hf: DTensor passthrough for DCP path (rename + transpose). Plain tensor init path unchanged (slice + transpose + create DTensor). - convert_single_tensor_to_hf: removed to_local() and .to(dtype). Just rename + transpose, DTensors pass through. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: HuiyingLi <willwin.lee@gmail.com>
HuiyingLi
force-pushed
the
huiyingl/perf-qwen3_5_moe-simplify-state-dict-adapter
branch
from
45e0e40 to
389899b
Compare
Contributor
Author
|
/ok to test 389899b |
Contributor
|
/claude review |
hemildesai
approved these changes
Mar 23, 2026
HuiyingLi
deleted the
huiyingl/perf-qwen3_5_moe-simplify-state-dict-adapter
branch
torsli
pushed a commit
that referenced
this pull request
Mar 24, 2026
…1589) perf: simplify Qwen3.5-MoE state_dict_adapter to rename+transpose only Same fix as PR #1570 for Qwen3-VL-MoE, applied to Qwen3.5-MoE: - to_hf: removed all_gather_object, full_tensor(), per-expert CPU copies, .to(dtype) casts. Now just renames keys + transpose(1,2). No comms. - from_hf: DTensor passthrough for DCP path (rename + transpose). Plain tensor init path unchanged (slice + transpose + create DTensor). - convert_single_tensor_to_hf: removed to_local() and .to(dtype). Just rename + transpose, DTensors pass through. Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
linnanwang
pushed a commit
that referenced
this pull request
Apr 24, 2026
…1589) perf: simplify Qwen3.5-MoE state_dict_adapter to rename+transpose only Same fix as PR #1570 for Qwen3-VL-MoE, applied to Qwen3.5-MoE: - to_hf: removed all_gather_object, full_tensor(), per-expert CPU copies, .to(dtype) casts. Now just renames keys + transpose(1,2). No comms. - from_hf: DTensor passthrough for DCP path (rename + transpose). Plain tensor init path unchanged (slice + transpose + create DTensor). - convert_single_tensor_to_hf: removed to_local() and .to(dtype). Just rename + transpose, DTensors pass through. Signed-off-by: HuiyingLi <willwin.lee@gmail.com> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Applies the same optimization as #1570 (Qwen3-VL-MoE) to the Qwen3.5-MoE state_dict_adapter, which had the identical unnecessary complexity.
Changes
to_hf: Removedall_gather_object,full_tensor(), per-expert CPU copies,.to(dtype)casts, and thedevice_meshbranch (~85 lines removed). Now just renames keys +transpose(1, 2). No comms needed — DCP handles distributed save.from_hf: Added DTensor vs plain tensor path distinction:create_dtensor_from_local.convert_single_tensor_to_hf: Removedto_local()and.to(self.dtype)calls. Simple key rename + transpose; DTensors pass through.import torch.distributed as dist(no longer needed).Benchmark (Qwen3.5-35B-A3B real weights, 1811 keys, 848 expert keys, single node)
from_hf(plain tensor)to_hf(no device_mesh)to_hf(with device_mesh)The main gain is eliminating the
to_hfhang with device_mesh (same issue as #1570) and removing unnecessary all-gather comms on the DCP save path.E2E validation (8x H100, Qwen3.5-35B-A3B real weights)
Test plan
🤖 Generated with Claude Code