Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce prefetch SM usage when using pipeline prefetching #2991

Closed
wants to merge 3 commits into from

Conversation

sryap
Copy link
Contributor

@sryap sryap commented Aug 14, 2024

Summary:
This diff limits the SM usage on masked_index_put and
masked_index_get when pipeline prefetching is used by using a small
grid size. This is to reduce the interference of these kernels with
the kernels on the compute stream during prefetch. The grid size is
currently set to 1/8 of the total number of SMs. However, this number
has to be tuned.

Differential Revision: D61145930

sarunya added 2 commits August 14, 2024 16:27
Differential Revision: D61294287
Differential Revision: D60727327
Copy link

netlify bot commented Aug 14, 2024

Deploy Preview for pytorch-fbgemm-docs ready!

Name Link
🔨 Latest commit 35c00b0
🔍 Latest deploy log https://app.netlify.com/sites/pytorch-fbgemm-docs/deploys/66bd8d0a46d80f0008ca97a2
😎 Deploy Preview https://deploy-preview-2991--pytorch-fbgemm-docs.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D61145930

@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D61145930

sryap added a commit to sryap/FBGEMM that referenced this pull request Aug 15, 2024
Summary:
Pull Request resolved: pytorch#2991

This diff limits the SM usage on `masked_index_put` and
`masked_index_get` when pipeline prefetching is used by using a small
grid size.  This is to reduce the interference of these kernels with
the kernels on the compute stream during prefetch.  The grid size is
currently set to 1/8 of the total number of SMs.  However, this number
has to be tuned.

Differential Revision: D61145930
Summary:
Pull Request resolved: pytorch#2991

This diff limits the SM usage on `masked_index_put` and
`masked_index_get` when pipeline prefetching is used by using a small
grid size.  This is to reduce the interference of these kernels with
the kernels on the compute stream during prefetch.  The grid size is
currently set to 1/8 of the total number of SMs.  However, this number
has to be tuned.

Differential Revision: D61145930
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D61145930

@facebook-github-bot
Copy link
Contributor

This pull request has been merged in 7a2ec83.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants