Skip to content

[ROCm] Autocast RNN Support #121539

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 1 commit into from
Closed

[ROCm] Autocast RNN Support #121539

wants to merge 1 commit into from

Conversation

alugorey
Copy link
Contributor

@alugorey alugorey commented Mar 8, 2024 •
edited by pytorch-bot bot
Loading

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Mar 8, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/121539

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (1 Unrelated Failure)

As of commit 6863e63 with merge base 75bb049 (image):

FLAKY - The following job failed but was likely due to flakiness present on trunk:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@pytorch-bot pytorch-bot bot added ciflow/rocm Trigger "default" config CI on ROCm module: rocm AMD GPU support for Pytorch labels Mar 8, 2024
@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Mar 8, 2024

Please seek CI approval before scheduling CIFlow labels

@pytorch-bot pytorch-bot bot removed the ciflow/rocm Trigger "default" config CI on ROCm label Mar 8, 2024
@alugorey alugorey mentioned this pull request Mar 8, 2024
Closed
albanD
albanD approved these changes Mar 11, 2024
View reviewed changes
Copy link
Collaborator

@albanD albanD left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good!

Will let @jeffdaily do a final review and trigger the merge

@albanD albanD added the triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module label Mar 11, 2024
@jeffdaily jeffdaily added ciflow/trunk Trigger trunk jobs on your pull request ciflow/rocm Trigger "default" config CI on ROCm labels Mar 11, 2024
@jeffdaily
Copy link
Collaborator

@pytorchbot merge

@pytorchmergebot
Copy link
Collaborator

Merge failed

Reason: This PR needs a release notes: label
If your changes are user facing and intended to be a part of release notes, please use a label starting with release notes:.

If not, please add the topic: not user facing label.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "topic: not user facing"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

Details for Dev Infra team Raised by workflow job

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Mar 11, 2024

Didn't find following labels among repository labels: release notes: Autocast support for MIOpen RNNs

@jeffdaily jeffdaily added the release notes: rocm mandatorylabel label Mar 11, 2024
@jeffdaily
Copy link
Collaborator

@pytorchbot merge

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Mar 11, 2024

Didn't find following labels among repository labels: release notes:

@jithunnair-amd
Copy link
Collaborator

@pytorchbot merge -f "Bot is confused about labels. CI is green"

@pytorchmergebot
Copy link
Collaborator

Merge started

Your change will be merged immediately since you used the force (-f) flag, bypassing any CI checks (ETA: 1-5 minutes). Please use -f as last resort and instead consider -i/--ignore-current to continue the merge ignoring current failures. This will allow currently pending tests to finish and report signal before the merge.

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging
Check the merge workflow status
here

@dnikolaev-amd dnikolaev-amd mentioned this pull request Aug 16, 2024
Merged
pruthvistony pushed a commit to ROCm/pytorch that referenced this pull request Aug 20, 2024
@dnikolaev-amd
[release/2.3] skip test_autocast_rnn (#1539)
e6c4aba 
Test was enabled in the upstream
pytorch#121539
But failed with "AssertionError: Tensor-likes are not close! Mismatched
elements: 30 / 30 (100.0%)"
in release/2.3 and rocm6.3_internal_testing on MI300 and passed on MI210
Pass with AMD_LOG_LEVEL=3 on MI300
Need to dig deeper
Skip TestCuda::test_autocast_rnn as in other releases
(https://ontrack-internal.amd.com/browse/SWDEV-468117)
release/2.1
https://github.com/ROCm/pytorch/blob/073f5dcaebc61b69a71de8918e574053bf95c40b/test/test_cuda.py#L2023
release/2.2
https://github.com/ROCm/pytorch/blob/a3208baad2f2d537f10362fe9ca30f60ff5aea21/test/test_cuda.py#L2047

Co-authored-by: root <root@banff-cyxtera-s80-2.amd.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jeffdaily jeffdaily jeffdaily approved these changes

@albanD albanD albanD approved these changes

@jithunnair-amd jithunnair-amd Awaiting requested review from jithunnair-amd jithunnair-amd is a code owner

Assignees

No one assigned

Labels

ciflow/rocm Trigger "default" config CI on ROCm ciflow/trunk Trigger trunk jobs on your pull request Merged module: rocm AMD GPU support for Pytorch open source release notes: rocm mandatorylabel triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Mixed precision is not working with CudnnLSTM on rocm

6 participants

@alugorey @jeffdaily @pytorchmergebot @jithunnair-amd @albanD @pytorchbot