[None][test] keep MXFP4 block-scale unswizzle on GPU in create_weights#15951
Conversation
MXFP4MXFP8QuantizeUtil.create_weights no longer moves fp4_quantize block scales to CPU before block_scale_interleave_reverse. The op already supports CUDA tensors, matching W4A8NVFP4FP8QuantizeUtil. Signed-off-by: xxi <xxi@nvidia.com>
📝 WalkthroughWalkthroughIn the MXFP4 test quantization utility, the weight creation logic for w1/w2/w3 block-scale tensors was changed to reshape directly on the current device, removing intermediate ChangesMXFP4 Scale Reshape Update
Estimated code review effort: 1 (Trivial) | ~3 minutes Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
|
/bot run --disable-fail-fast |
|
PR_Github #57904 [ run ] triggered by Bot. Commit: |
|
PR_Github #57904 [ run ] completed with state
|
|
/bot run --disable-fail-fast |
|
PR_Github #57939 [ run ] triggered by Bot. Commit: |
|
PR_Github #57939 [ run ] completed with state |
NVIDIA#15951) Signed-off-by: xxi <xxi@nvidia.com>
Summary
.cpu()beforeblock_scale_interleave_reverseinMXFP4MXFP8QuantizeUtil.create_weights(and inheritedMXFP4FP8QuantizeUtil).fp4_quantizealready produces GPU block scales;block_scale_interleave_reversesupports CUDA tensors, matchingW4A8NVFP4FP8QuantizeUtil.Test plan
test_moe_backendfiltered to W4A8_MXFP4_* quant algosSummary by CodeRabbit