[ROCm] Limit number of values per thread for reductions on three dimensions #2460
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Jenkins build for 53fbf7866d00ae012510c041cb5c3e13d3a1c214 commit finished as FAILURE |
Collaborator
Author
|
reproducer details at pytorch#159652 (comment) |
pruthvistony
approved these changes
Aug 6, 2025
Collaborator
pruthvistony
left a comment
pruthvistony
left a comment
There was a problem hiding this comment.
Please check for any regressing models, may need to rebase the numbers.
Collaborator
Author
|
! cherry-pick --onto release/2.8 |
okakarpa
pushed a commit
that referenced
this pull request
Aug 6, 2025
…nsions (#2460) In the current implementation of reductions in three dimensions for AMD GPUs the number of values per thread is unbounded and can end up being in the hundreds of thousands for certain tensors. This of course is bad for performance. This patch fixes this issue by increasing the parallelism and thus lowering the number of value per thread to reasonable limits i.e. less than 2048 values per thread. The performance gains can be between 10x-17x for certain examples where the number of values per thread was originally very high. cherry-pick of pytorch#159652
Collaborator
|
Created branch autogenerated/release/2.8_cherry-pick_pr-2460 and #2469 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In the current implementation of reductions in three dimensions for AMD GPUs the number of values per thread is unbounded and can end up being in the hundreds of thousands for certain tensors. This of course is bad for performance. This patch fixes this issue by increasing the parallelism and thus lowering the number of value per thread to reasonable limits i.e. less than 2048 values per thread. The performance gains can be between 10x-17x for certain examples where the number of values per thread was originally very high.
cherry-pick of pytorch#159652
Cherry-picked to release/2.8 branch via #2469