Fix softmax_backward_data cpu implementation error when argument output is noncontinguous #139740
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/139740
Note: Links to docs will display an error until the docs builds have been completed. ❗ 1 Active SEVsThere are 1 currently active SEVs. If your PR is affected, please view them below: ✅ No FailuresAs of commit 404a078 with merge base 12d225d ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
|
@pytorchbot label "topic: not user facing" |
|
@pytorchbot rebase |
|
You don't have permissions to rebase this PR since you are a first time contributor. If you think this is a mistake, please contact PyTorch Dev Infra. |
|
@pytorchbot drci |
|
Cc @scw @svenstaro @JackDanger This pull request encounters a check label failure, and the error message is unclear. Can you assist me? Thank you. |
|
@lixin-sxty why ping me? I have nothing to do with this project. |
Sorry for the misunderstanding; when I typed |
lixin-sxty
changed the title
…ontinguous Signed-off-by: xin.li <xin.li@enflame-tech.com>
|
@lixin-sxty can you add a test, maybe in |
bdhirsh
added
the
triaged
lixin-sxty
changed the title
I will check the tests for softmax_backward_data in common_methods_invocations.py and add a test case if necessary. |
lixin-sxty
changed the title
|
According to https://github.com/pytorch/pytorch/blob/main/torch/testing/_internal/common_methods_invocations.py#L14538, the test_noncontiguous_samples was failing on CPU. This case now passes with the fix, so I have removed this flag. @bdhirsh |
lixin-sxty
requested review from
Chillee,
kshitij12345 and
zou3519
as code owners
|
@pytorchbot merge |
Merge startedYour change will be merged once all checks pass (ETA 0-4 Hours). Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
…ut is noncontinguous (pytorch#139740) Implementation of the `softmax_backward_data` operator for the CPU backend produces incorrect results when the `output` argument is non-contiguous. Here is a test case that demonstrates this issue: ```python torch.manual_seed(0) op = torch.ops.aten._softmax_backward_data grad_output = torch.ones(3, 3, 3) temp = torch.randn(3, 10, 3) out = temp[:, :3, :] out = out.contiguous() print(out.is_contiguous()) grad_input = op(grad_output, out, 1, torch.float32) print(grad_input) ``` In this test case, the variable `grad_input` yields incorrect results if the line `out = out.contiguous()` is commented out. With this fix, `grad_input` consistently produces the same results whenever `output` is contiguous. Pull Request resolved: pytorch#139740 Approved by: https://github.com/zou3519
6 participants
Implementation of the
softmax_backward_dataoperator for the CPU backend produces incorrect results when theoutputargument is non-contiguous.Here is a test case that demonstrates this issue:
In this test case, the variable
grad_inputyields incorrect results if the lineout = out.contiguous()is commented out. With this fix,grad_inputconsistently produces the same results wheneveroutputis contiguous.