Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use more performant bsr_scatter_mm within bsr_dense_mm when blocksize is 16. #111489

Closed
wants to merge 2 commits into from
Closed

Use more performant bsr_scatter_mm within bsr_dense_mm when blocksize is 16. #111489

wants to merge 2 commits into from

Conversation

pearu
Copy link
Collaborator

@pearu pearu commented Oct 18, 2023 •
edited

This was referenced Oct 18, 2023
Closed
Closed
@pytorch-bot pytorch-bot bot added the release notes: sparse release notes category label Oct 18, 2023
@pytorch-bot
Copy link

pytorch-bot bot commented Oct 18, 2023 •
edited

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/111489

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 4d751ed with merge base 57c7aa1 (image):
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

pearu added a commit that referenced this pull request Oct 18, 2023
@pearu
Use more performant bsr_scatter_mm within bsr_dense_mm when blocksize…
7d95766 
… is 16.

ghstack-source-id: 7c28a7bf9c372e2c3f3a47e2b9491f839306a5ac
Pull Request resolved: #111489
@pearu pearu mentioned this pull request Oct 22, 2023
Closed
@pearu pearu mentioned this pull request Oct 23, 2023
Closed
pytorchmergebot pushed a commit that referenced this pull request Oct 23, 2023
@pearu @pytorchmergebot
Add NVIDIA A100 optimized meta parameters to bsr_dense_mm (#111760)
6382011 
As in the title.

The figures below illustrate the performance differences of bsr_dense_mm with optimized parameters and bsr_dense_mm with default parameters (GPU: NVIDIA A100-SXM4-80GB). The first figure represents the performance equilibrium point in BSR tensor sparsity at which value bsr_dense_mm have the same performance characteristics as torch.matmul. The second figure represents speedups from using optimized meta parameters in bsr_dense_mm at its performance equilibrium points with respect to bsr_dense_mm with default meta parameters.

In sum, this PR speeds up `bsr_dense_mm` about 50 % depending on the bsr tensor shape and blocksize and lowers the performance equilibrium points of BSR tensor sparsity and strided tensor for matmul operations.

<img src="https://github.com/pytorch/pytorch/assets/402156/6fe9d35f-dd21-4aa0-bb01-6ee257254453" width="48%"> <img src="https://github.com/pytorch/pytorch/assets/402156/506921c6-3770-4209-ad3d-498d2ae4989d" width="48%">

Pull Request resolved: #111760
Approved by: https://github.com/cpuhrsch
ghstack dependencies: #110396, #111470, #111489
pytorchmergebot pushed a commit that referenced this pull request Oct 23, 2023
@pearu @pytorchmergebot
Add batched dimensions support to the second operand of bsr_scatter_mm (
b969c67 
#111796)

Pull Request resolved: #111796
Approved by: https://github.com/cpuhrsch
ghstack dependencies: #110396, #111470, #111489, #111760
andreigh pushed a commit to andreigh/pytorch that referenced this pull request Oct 26, 2023
@pearu @andreigh
Use more performant bsr_scatter_mm within bsr_dense_mm when blocksize…
679aa5a 
… is 16. (pytorch#111489)

Pull Request resolved: pytorch#111489
Approved by: https://github.com/cpuhrsch
ghstack dependencies: pytorch#110396, pytorch#111470
andreigh pushed a commit to andreigh/pytorch that referenced this pull request Oct 26, 2023
@pearu @andreigh
Add NVIDIA A100 optimized meta parameters to bsr_dense_mm (pytorch#11…
9b0faf0 
…1760)

As in the title.

The figures below illustrate the performance differences of bsr_dense_mm with optimized parameters and bsr_dense_mm with default parameters (GPU: NVIDIA A100-SXM4-80GB). The first figure represents the performance equilibrium point in BSR tensor sparsity at which value bsr_dense_mm have the same performance characteristics as torch.matmul. The second figure represents speedups from using optimized meta parameters in bsr_dense_mm at its performance equilibrium points with respect to bsr_dense_mm with default meta parameters.

In sum, this PR speeds up `bsr_dense_mm` about 50 % depending on the bsr tensor shape and blocksize and lowers the performance equilibrium points of BSR tensor sparsity and strided tensor for matmul operations.

<img src="https://github.com/pytorch/pytorch/assets/402156/6fe9d35f-dd21-4aa0-bb01-6ee257254453" width="48%"> <img src="https://github.com/pytorch/pytorch/assets/402156/506921c6-3770-4209-ad3d-498d2ae4989d" width="48%">

Pull Request resolved: pytorch#111760
Approved by: https://github.com/cpuhrsch
ghstack dependencies: pytorch#110396, pytorch#111470, pytorch#111489
andreigh pushed a commit to andreigh/pytorch that referenced this pull request Oct 26, 2023
@pearu @andreigh
Add batched dimensions support to the second operand of bsr_scatter_mm (
6d6a101 
pytorch#111796)

Pull Request resolved: pytorch#111796
Approved by: https://github.com/cpuhrsch
ghstack dependencies: pytorch#110396, pytorch#111470, pytorch#111489, pytorch#111760
@facebook-github-bot facebook-github-bot deleted the gh/pearu/122/head branch October 27, 2023 14:25
xuhancn pushed a commit to xuhancn/pytorch that referenced this pull request Nov 7, 2023
@pearu @xuhancn
Use more performant bsr_scatter_mm within bsr_dense_mm when blocksize…
4b96070 
… is 16. (pytorch#111489)

Pull Request resolved: pytorch#111489
Approved by: https://github.com/cpuhrsch
ghstack dependencies: pytorch#110396, pytorch#111470
xuhancn pushed a commit to xuhancn/pytorch that referenced this pull request Nov 7, 2023
@pearu @xuhancn
Add NVIDIA A100 optimized meta parameters to bsr_dense_mm (pytorch#11…
0a54f8b 
…1760)

As in the title.

The figures below illustrate the performance differences of bsr_dense_mm with optimized parameters and bsr_dense_mm with default parameters (GPU: NVIDIA A100-SXM4-80GB). The first figure represents the performance equilibrium point in BSR tensor sparsity at which value bsr_dense_mm have the same performance characteristics as torch.matmul. The second figure represents speedups from using optimized meta parameters in bsr_dense_mm at its performance equilibrium points with respect to bsr_dense_mm with default meta parameters.

In sum, this PR speeds up `bsr_dense_mm` about 50 % depending on the bsr tensor shape and blocksize and lowers the performance equilibrium points of BSR tensor sparsity and strided tensor for matmul operations.

<img src="https://github.com/pytorch/pytorch/assets/402156/6fe9d35f-dd21-4aa0-bb01-6ee257254453" width="48%"> <img src="https://github.com/pytorch/pytorch/assets/402156/506921c6-3770-4209-ad3d-498d2ae4989d" width="48%">

Pull Request resolved: pytorch#111760
Approved by: https://github.com/cpuhrsch
ghstack dependencies: pytorch#110396, pytorch#111470, pytorch#111489
xuhancn pushed a commit to xuhancn/pytorch that referenced this pull request Nov 7, 2023
@pearu @xuhancn
Add batched dimensions support to the second operand of bsr_scatter_mm (
c0432b5 
pytorch#111796)

Pull Request resolved: pytorch#111796
Approved by: https://github.com/cpuhrsch
ghstack dependencies: pytorch#110396, pytorch#111470, pytorch#111489, pytorch#111760
Skylion007 pushed a commit to Skylion007/pytorch that referenced this pull request Nov 14, 2023
@pearu @Skylion007
Use more performant bsr_scatter_mm within bsr_dense_mm when blocksize…
b4b6873 
… is 16. (pytorch#111489)

Pull Request resolved: pytorch#111489
Approved by: https://github.com/cpuhrsch
ghstack dependencies: pytorch#110396, pytorch#111470
Skylion007 pushed a commit to Skylion007/pytorch that referenced this pull request Nov 14, 2023
@pearu @Skylion007
Add NVIDIA A100 optimized meta parameters to bsr_dense_mm (pytorch#11…
5ad8abe 
…1760)

As in the title.

The figures below illustrate the performance differences of bsr_dense_mm with optimized parameters and bsr_dense_mm with default parameters (GPU: NVIDIA A100-SXM4-80GB). The first figure represents the performance equilibrium point in BSR tensor sparsity at which value bsr_dense_mm have the same performance characteristics as torch.matmul. The second figure represents speedups from using optimized meta parameters in bsr_dense_mm at its performance equilibrium points with respect to bsr_dense_mm with default meta parameters.

In sum, this PR speeds up `bsr_dense_mm` about 50 % depending on the bsr tensor shape and blocksize and lowers the performance equilibrium points of BSR tensor sparsity and strided tensor for matmul operations.

<img src="https://github.com/pytorch/pytorch/assets/402156/6fe9d35f-dd21-4aa0-bb01-6ee257254453" width="48%"> <img src="https://github.com/pytorch/pytorch/assets/402156/506921c6-3770-4209-ad3d-498d2ae4989d" width="48%">

Pull Request resolved: pytorch#111760
Approved by: https://github.com/cpuhrsch
ghstack dependencies: pytorch#110396, pytorch#111470, pytorch#111489
Skylion007 pushed a commit to Skylion007/pytorch that referenced this pull request Nov 14, 2023
@pearu @Skylion007
Add batched dimensions support to the second operand of bsr_scatter_mm (
b9e61bd 
pytorch#111796)

Pull Request resolved: pytorch#111796
Approved by: https://github.com/cpuhrsch
ghstack dependencies: pytorch#110396, pytorch#111470, pytorch#111489, pytorch#111760
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@cpuhrsch cpuhrsch cpuhrsch approved these changes

@amjames amjames Awaiting requested review from amjames

Assignees
No one assigned
Labels
Merged open source release notes: sparse release notes category
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@pearu @cpuhrsch @pytorchbot @pytorchmergebot