Skip to content

[None][test] keep MXFP4 block-scale unswizzle on GPU in create_weights#15951

Merged
xxi-nv merged 1 commit into
NVIDIA:mainfrom
xxi-nv:user/xxi/mxfp4-block-scale-gpu-opt
Jul 7, 2026
Merged

[None][test] keep MXFP4 block-scale unswizzle on GPU in create_weights#15951
xxi-nv merged 1 commit into
NVIDIA:mainfrom
xxi-nv:user/xxi/mxfp4-block-scale-gpu-opt

Conversation

@xxi-nv

@xxi-nv xxi-nv commented Jul 6, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Remove redundant .cpu() before block_scale_interleave_reverse in MXFP4MXFP8QuantizeUtil.create_weights (and inherited MXFP4FP8QuantizeUtil).
  • fp4_quantize already produces GPU block scales; block_scale_interleave_reverse supports CUDA tensors, matching W4A8NVFP4FP8QuantizeUtil.

Test plan

  • OCI GB200: test_moe_backend filtered to W4A8_MXFP4_* quant algos
  • CI

Summary by CodeRabbit

  • Tests
    • Improved quantization test coverage for block-scale handling.
    • Updated scale reshaping behavior to better match expected results during weight processing.

MXFP4MXFP8QuantizeUtil.create_weights no longer moves fp4_quantize block scales to CPU before block_scale_interleave_reverse. The op already supports CUDA tensors, matching W4A8NVFP4FP8QuantizeUtil.

Signed-off-by: xxi <xxi@nvidia.com>
@coderabbitai

coderabbitai Bot commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

In the MXFP4 test quantization utility, the weight creation logic for w1/w2/w3 block-scale tensors was changed to reshape directly on the current device, removing intermediate .cpu() calls before view().

Changes

MXFP4 Scale Reshape Update

Layer / File(s) Summary
Remove CPU move before reshape
tests/unittest/_torch/modules/moe/quantize_utils.py
w1/w2/w3_sf_block_unswizzled tensors are now reshaped directly via block_scale_interleave_reverse(...).view(...) without first moving to CPU.

Estimated code review effort: 1 (Trivial) | ~3 minutes

Possibly related PRs

  • NVIDIA/TensorRT-LLM#15714: Also modifies the same MoE MXFP4 quantization test utility to adjust scale tensor handling and reshaping for GPU processing.

Suggested reviewers: leslie-fang25

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly reflects the GPU unswizzle change in create_weights and is concise.
Description check ✅ Passed The description summarizes the change and test plan, and only omits the optional PR Checklist section.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@xxi-nv xxi-nv requested a review from leslie-fang25 July 7, 2026 02:46
@xxi-nv

xxi-nv commented Jul 7, 2026

Copy link
Copy Markdown
Collaborator Author

/bot run --disable-fail-fast

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #57904 [ run ] triggered by Bot. Commit: c517357 Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #57904 [ run ] completed with state FAILURE. Commit: c517357
/LLM/main/L0_MergeRequest_PR pipeline #46598 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

  • Please check the failed tests and fix your PR
  • If you cannot view the failures, ask the CI triggerer to share details
  • Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

@xxi-nv

xxi-nv commented Jul 7, 2026

Copy link
Copy Markdown
Collaborator Author

/bot run --disable-fail-fast

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #57939 [ run ] triggered by Bot. Commit: c517357 Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #57939 [ run ] completed with state SUCCESS. Commit: c517357
/LLM/main/L0_MergeRequest_PR pipeline #46624 completed with status: 'SUCCESS'

CI Report

Link to invocation

@xxi-nv xxi-nv merged commit 33526e6 into NVIDIA:main Jul 7, 2026
12 of 14 checks passed
@xxi-nv xxi-nv deleted the user/xxi/mxfp4-block-scale-gpu-opt branch July 7, 2026 07:15
BrianLi23 pushed a commit to BrianLi23/TensorRT-LLM that referenced this pull request Jul 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants