Use to() instead of contiguous() to generate channels last tensor for Intel XPU #161041
Conversation
pytorch-bot
bot
added
the
module: cpu
ZhiweiYan-96
added
topic: not user facing
|
@NeoZhangJianyu , you can invoke |
as_strided() is used to set the new stride to tensor. But there is no existed single function to create the correct stride of a tensor in current pytorch code. So, it's very hard to call as_strided() to correct the stride. Thank you! |
|
Updated the solution description: Refer to the solution of #96791, use to() to correct the stride for channel last case. Note: to() can't replace the contiguous() to make the tensor weights are contiguous. Refer to https://github.com/pytorch/pytorch/blob/main/aten/src/ATen/native/cudnn/ConvShared.cpp#L772 |
|
@NeoZhangJianyu , could you help rebase this PR and fix the linter issue? |
There was a problem hiding this comment.
Pull Request Overview
Fixes an issue where convolution backward operations on Intel XPU fail with tensors having abnormal stride patterns. The solution addresses cases where output gradient tensors have incorrect stride orders that lead to wrong results.
- Adds stride validation to detect non-decreasing stride patterns
- Uses
to()method to correct stride for channel-last format before falling back tocontiguous() - Includes comprehensive test case to verify the fix
Reviewed Changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| aten/src/ATen/native/mkldnn/xpu/Conv.cpp | Implements stride validation function and applies stride correction logic in convolution backward |
| test/xpu/test_conv.py | Adds test case that creates abnormal stride patterns and verifies backward pass correctness |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
|
|
||
| if (!is_stride_decrease_order(grad_output_)) | ||
| grad_output_ = grad_output_.to(mfmt); | ||
|
|
There was a problem hiding this comment.
[nitpick] There are unnecessary blank lines (618 and 621) that break the logical flow of the conditional checks. Remove these extra blank lines to improve code readability.
| if (!is_stride_decrease_order(grad_output_)) | |
| grad_output_ = grad_output_.to(mfmt); | |
| if (!is_stride_decrease_order(grad_output_)) | |
| grad_output_ = grad_output_.to(mfmt); |
|
Looks like this PR hasn't been updated in a while so we're going to go ahead and mark this as |
5 participants
Fixes #95693 for Intel XPU case.
In same cases of Conv Backward, the output grad tensor's stride will be abnormal format.
Like output_grad.shape = torch.Size([1, 1, 2, 2])
Normal stride: [4, 4, 2, 1]
Abnormal stride: [4, 1, 2, 1]
That will lead to the wrong result of conv backward for Intel XPU.
Solution:
Refer to the solution of Use .to instead of contiguous to generate channels last tensor #96791, use to() to correct the stride for channel last case.
Note: to() can't replace the contiguous() to make the tensor weights are contiguous. Refer to https://github.com/pytorch/pytorch/blob/main/aten/src/ATen/native/cudnn/ConvShared.cpp#L772
Add unit test case.
This PR is rebased on old PR #160606 with more review comments.
cc @jgong5 @mingfeima @XiaobingSuper @sanchitintel @ashokei @jingxu10 @jerryzh168 @aditew01 @gujinghui @EikanWang @fengyuan14 @guangyey