-
Notifications
You must be signed in to change notification settings - Fork 26.1k
add copy_ support for float4 dtype #169595
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Summary: Enables `copy_` support for the `torch.float4_e2m1fn_x2` dtype. This is useful when slicing a tensor across dim1 and then calling contiguous, which can happen in vllm and therefore should be supported. Test Plan: ``` pytest test/quantization/core/experimental/test_floatx.py -s -k test_float4_e2m1fn_x2 ``` Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned]
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/169595
Note: Links to docs will display an error until the docs builds have been completed. ✅ You can merge normally! (7 Unrelated Failures)As of commit 257cef6 with merge base 685ba6b ( UNSTABLE - The following jobs are marked as unstable, possibly due to flakiness on trunk:
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
Summary: Enables `copy_` support for the `torch.float4_e2m1fn_x2` dtype. This is useful when slicing a tensor across dim1 and then calling contiguous, which can happen in vllm and therefore should be supported. Test Plan: ``` pytest test/quantization/core/experimental/test_floatx.py -s -k test_float4_e2m1fn_x2 ``` Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 4db38fd Pull Request resolved: #169595
Summary: Enables `copy_` support for the `torch.float4_e2m1fn_x2` dtype. This is useful when slicing a tensor across dim1 and then calling contiguous, which can happen in vllm and therefore should be supported. Test Plan: ``` pytest test/quantization/core/experimental/test_floatx.py -s -k test_float4_e2m1fn_x2 ``` Reviewers: Subscribers: Tasks: Tags: cc jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 jerryzh168 aditew01 [ghstack-poisoned]
Summary: Enables `copy_` support for the `torch.float4_e2m1fn_x2` dtype. This is useful when slicing a tensor across dim1 and then calling contiguous, which can happen in vllm and therefore should be supported. Test Plan: ``` pytest test/quantization/core/experimental/test_floatx.py -s -k test_float4_e2m1fn_x2 ``` Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: d1315b2 Pull Request resolved: #169595
Summary: Uses the `torch.float4_e2m1fn_x2` dtype in mxfp4 and nxfp4 torchao tensors. Requires pytorch/pytorch#169595, so we need to wait for the next PyTorch branch cut. Note: nvfp4 models in vllm currently hit an error https://gist.github.com/vkuzo/e1407ee68c9ebb8d0f67478aedd81b96 before or after this PR, so some more debugging to do before landing this. Test Plan: ``` CUDA_VISIBLE_DEVICES=6 time pytest test/prototype/mx_formats/ -s ``` Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: ee5d3a0 ghstack-comment-id: 3614417747 Pull-Request: #3440
|
@pytorchbot merge -f |
|
❌ 🤖 pytorchbot command failed: Try |
|
@pytorchbot merge -f "failures unrelated" |
Merge startedYour change will be merged immediately since you used the force (-f) flag, bypassing any CI checks (ETA: 1-5 minutes). Please use Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
Stack from ghstack (oldest at bottom):
Summary:
Enables
copy_support for thetorch.float4_e2m1fn_x2dtype. This isuseful when slicing a tensor across dim1 and then calling contiguous,
which can happen in vllm and therefore should be supported.
Test Plan:
Reviewers:
Subscribers:
Tasks:
Tags:
cc @jgong5 @mingfeima @XiaobingSuper @sanchitintel @ashokei @jingxu10 @jerryzh168 @aditew01