Allow add_persistent_r_block to scale up rblock up to a limit #162296

PaulZhang12 · 2025-09-05T20:39:46Z

Stack from ghstack (oldest at bottom):

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @ipiszy @chenyang78 @kadeng @muchulee8 @amjames @chauhang @aakhundov @coconutruben

[ghstack-poisoned]

pytorch-bot · 2025-09-05T20:39:50Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/162296

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit the bot commands wiki

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

torch.distributed failures on windows CI build

✅ No Failures

As of commit 09a5b18 with merge base 456fbea ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

ghstack-source-id: 10eca5d Pull Request resolved: #162296

…mit" cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov coconutruben [ghstack-poisoned]

PaulZhang12 · 2025-09-10T18:16:53Z

@PaulZhang12 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

PaulZhang12 · 2025-09-10T18:38:33Z

@PaulZhang12 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

PaulZhang12 · 2025-09-10T18:39:12Z

@PaulZhang12 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

PaulZhang12 · 2025-09-15T18:59:04Z

@PaulZhang12 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

…mit" cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov coconutruben Differential Revision: [D82132959](https://our.internmc.facebook.com/intern/diff/D82132959) [ghstack-poisoned]

eellison

did this help any particular kernel?

eellison · 2025-09-18T17:33:25Z

torch/_inductor/codegen/triton.py

                # large rblock inhibits xblock size, dont attempt if there is a decent amount of
                # reads coalesced by xblock
                and r_coalesce_ratio >= 8.0
-                # TODO - need more detailed register analysis


did you see any speedups on rblock == 65536 ?

edit: we still limit to 32768 below..

Ah yes let me update the torch.compile vs quack results. For large r, we see improvement between final heuristic and just torch._dynamo.reset with this

eellison · 2025-09-18T18:54:11Z

torch/_inductor/runtime/triton_heuristics.py

        c = make_config(
            xnumel,
-            rnumel,
+            min(rnumel, 32768),


ah, okay, following.

@eellison yeah above 32768 we don't see any benefit

PaulZhang12 · 2025-09-18T20:53:54Z

@pytorchbot merge

pytorchmergebot · 2025-09-18T20:55:47Z

Merge failed

Reason: This PR has internal changes and must be landed via Phabricator! Please try reimporting/rexporting the PR!

Details for Dev Infra team

Raised by workflow job

PaulZhang12 · 2025-09-22T18:42:34Z

@PaulZhang12 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

PaulZhang12 · 2025-09-22T21:21:19Z

@pytorchbot merge

pytorchmergebot · 2025-09-22T21:23:07Z

Merge failed

Reason: This PR has internal changes and must be landed via Phabricator! Please try reimporting/rexporting the PR!

Details for Dev Infra team

Raised by workflow job

PaulZhang12 · 2025-09-22T21:40:00Z

@pytorchbot merge -f "Ignore internal sync"

pytorchmergebot · 2025-09-22T21:41:31Z

Merge started

Your change will be merged immediately since you used the force (-f) flag, bypassing any CI checks (ETA: 1-5 minutes). Please use -f as last resort and instead consider -i/--ignore-current to continue the merge ignoring current failures. This will allow currently pending tests to finish and report signal before the merge.

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

…on x block (#162446) Scale up XBLOCK for contiguous persistent reductions based on rnumel and number of loads + stores <img width="928" height="656" alt="Screenshot 2025-09-18 at 5 02 57 PM" src="https://github.com/user-attachments/assets/ec3c561f-2a3f-4459-9e14-653715898da3" /> Differential Revision: [](https://our.internmc.facebook.com/intern/diff/) Differential Revision: [](https://our.internmc.facebook.com/intern/diff/) Pull Request resolved: #162446 Approved by: https://github.com/v0i0, https://github.com/eellison, https://github.com/shunting314 ghstack dependencies: #162296

…h#162296) <img width="654" height="392" alt="Screenshot 2025-09-18 at 4 22 53 PM" src="https://github.com/user-attachments/assets/975650ec-f769-43a6-bdf5-2885a8d40d3c" /> Pull Request resolved: pytorch#162296 Approved by: https://github.com/eellison

…on x block (pytorch#162446) Scale up XBLOCK for contiguous persistent reductions based on rnumel and number of loads + stores <img width="928" height="656" alt="Screenshot 2025-09-18 at 5 02 57 PM" src="https://github.com/user-attachments/assets/ec3c561f-2a3f-4459-9e14-653715898da3" /> Differential Revision: [](https://our.internmc.facebook.com/intern/diff/) Differential Revision: [](https://our.internmc.facebook.com/intern/diff/) Pull Request resolved: pytorch#162446 Approved by: https://github.com/v0i0, https://github.com/eellison, https://github.com/shunting314 ghstack dependencies: pytorch#162296

…on x block (#162446) Scale up XBLOCK for contiguous persistent reductions based on rnumel and number of loads + stores <img width="928" height="656" alt="Screenshot 2025-09-18 at 5 02 57 PM" src="https://github.com/user-attachments/assets/ec3c561f-2a3f-4459-9e14-653715898da3" /> Differential Revision: [](https://our.internmc.facebook.com/intern/diff/) Differential Revision: [](https://our.internmc.facebook.com/intern/diff/) Pull Request resolved: #162446 Approved by: https://github.com/v0i0, https://github.com/eellison, https://github.com/shunting314 ghstack dependencies: #162296

…on x block (pytorch#162446) Scale up XBLOCK for contiguous persistent reductions based on rnumel and number of loads + stores <img width="928" height="656" alt="Screenshot 2025-09-18 at 5 02 57 PM" src="https://github.com/user-attachments/assets/ec3c561f-2a3f-4459-9e14-653715898da3" /> Differential Revision: [](https://our.internmc.facebook.com/intern/diff/) Differential Revision: [](https://our.internmc.facebook.com/intern/diff/) Pull Request resolved: pytorch#162446 Approved by: https://github.com/v0i0, https://github.com/eellison, https://github.com/shunting314 ghstack dependencies: pytorch#162296

Allow add_persistent_r_block to scale up rblock up to a limit

7e319a7

[ghstack-poisoned]

pytorch-bot bot added ciflow/inductor module: inductor labels Sep 5, 2025

PaulZhang12 added a commit that referenced this pull request Sep 5, 2025

Allow add_persistent_r_block to scale up rblock up to a limit

98f6006

ghstack-source-id: 10eca5d Pull Request resolved: #162296

This was referenced Sep 9, 2025

Add num_store to inductor_meta and use it to scale persistent reduction x block #162446

Closed

Add less warps config to inner reductions #162447

Closed

PaulZhang12 added the topic: not user facing topic category label Sep 9, 2025

Update on "Allow add_persistent_r_block to scale up rblock up to a li…

053f2bf

…mit" cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov coconutruben [ghstack-poisoned]

pytorch-bot bot added the ciflow/trunk Trigger trunk jobs on your pull request label Sep 10, 2025

PaulZhang12 added 2 commits September 16, 2025 11:05

eellison approved these changes Sep 18, 2025

View reviewed changes

pytorchmergebot added the merging label Sep 18, 2025

pytorchmergebot removed the merging label Sep 18, 2025

pytorchmergebot added the merging label Sep 22, 2025

pytorchmergebot removed the merging label Sep 22, 2025

pytorchmergebot added the merging label Sep 22, 2025

pytorchmergebot closed this in 2b03663 Sep 22, 2025

pytorchmergebot added Merged and removed merging labels Sep 22, 2025

github-actions bot deleted the gh/PaulZhang12/24/head branch October 23, 2025 02:12

Allow add_persistent_r_block to scale up rblock up to a limit #162296

Allow add_persistent_r_block to scale up rblock up to a limit #162296

Uh oh!

Conversation

PaulZhang12 commented Sep 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot bot commented Sep 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/162296

❗ 1 Active SEVs

✅ No Failures

Uh oh!

PaulZhang12 commented Sep 10, 2025

Uh oh!

PaulZhang12 commented Sep 10, 2025

Uh oh!

PaulZhang12 commented Sep 10, 2025

Uh oh!

PaulZhang12 commented Sep 15, 2025

Uh oh!

eellison left a comment

Choose a reason for hiding this comment

Uh oh!

eellison Sep 18, 2025

Choose a reason for hiding this comment

Uh oh!

PaulZhang12 Sep 18, 2025

Choose a reason for hiding this comment

Uh oh!

eellison Sep 18, 2025

Choose a reason for hiding this comment

Uh oh!

PaulZhang12 Sep 18, 2025

Choose a reason for hiding this comment

Uh oh!

PaulZhang12 commented Sep 18, 2025

Uh oh!

pytorchmergebot commented Sep 18, 2025

Merge failed

Uh oh!

PaulZhang12 commented Sep 22, 2025

Uh oh!

PaulZhang12 commented Sep 22, 2025

Uh oh!

pytorchmergebot commented Sep 22, 2025

Merge failed

Uh oh!

PaulZhang12 commented Sep 22, 2025

Uh oh!

pytorchmergebot commented Sep 22, 2025

Merge started

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

PaulZhang12 commented Sep 5, 2025 •

edited

Loading

pytorch-bot bot commented Sep 5, 2025 •

edited

Loading