[inductor] no-side-effect codegen #107617

shunting314 · 2023-08-21T19:27:42Z

Stack from ghstack (oldest at bottom):

Inductor kernel codegen previously have the following side effect:

in Kernel.__exit__ , we add local used buffers in graph.removed_buffers
during codegen, we do memory allocation/free.

These cause doing multiple versions of codegen for the same kernel hard. The PR refactor the code to make kernel codegen not changing graph level states. After codegening a kernel, the graph level state is not changed so we can go on to codegen another version of the kernel if we want.

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @ngimel @yf225 @chenyang78 @kadeng @muchulee8 @aakhundov

[ghstack-poisoned]

pytorch-bot · 2023-08-21T19:27:45Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/107617

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit e11cf52 with merge base 39130c7 ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

ghstack-source-id: 03e8e58847a16268605f7a3156dfaf3785f6d9de Pull Request resolved: #107617

Inductor kernel codegen previously have the following side effect: - in `Kernel.__exit__ `, we add local used buffers in graph.removed_buffers - during codegen, we do memory allocation/free. These cause doing multiple versions of codegen for the same kernel hard. The PR refactor the code to make kernel codegen not changing graph level states. After codegening a kernel, the graph level state is not changed so we can go on to codegen another version of the kernel if we want. cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx peterbell10 ipiszy ngimel yf225 chenyang78 kadeng muchulee8 aakhundov [ghstack-poisoned]

ghstack-source-id: 5edd38d351ba5f15d390de5d890ca6eb29215249 Pull Request resolved: #107617

Inductor kernel codegen previously have the following side effect: - in `Kernel.__exit__ `, we add local used buffers in graph.removed_buffers - during codegen, we do memory allocation/free. These cause doing multiple versions of codegen for the same kernel hard. The PR refactor the code to make kernel codegen not changing graph level states. After codegening a kernel, the graph level state is not changed so we can go on to codegen another version of the kernel if we want. cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx peterbell10 ipiszy ngimel yf225 chenyang78 kadeng muchulee8 aakhundov [ghstack-poisoned]

ghstack-source-id: 236c6902f23f217cb6d36c8a70dc622e98a82cf2 Pull Request resolved: #107617

Inductor kernel codegen previously have the following side effect: - in `Kernel.__exit__ `, we add local used buffers in graph.removed_buffers - during codegen, we do memory allocation/free. These cause doing multiple versions of codegen for the same kernel hard. The PR refactor the code to make kernel codegen not changing graph level states. After codegening a kernel, the graph level state is not changed so we can go on to codegen another version of the kernel if we want. cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx peterbell10 ipiszy ngimel yf225 chenyang78 kadeng muchulee8 aakhundov [ghstack-poisoned]

ghstack-source-id: a20bfba952cb862e564e984c6b1ca248ccdbc5f8 Pull Request resolved: #107617

Inductor kernel codegen previously have the following side effect: - in `Kernel.__exit__ `, we add local used buffers in graph.removed_buffers - during codegen, we do memory allocation/free. These cause doing multiple versions of codegen for the same kernel hard. The PR refactor the code to make kernel codegen not changing graph level states. After codegening a kernel, the graph level state is not changed so we can go on to codegen another version of the kernel if we want. cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx peterbell10 ipiszy ngimel yf225 chenyang78 kadeng muchulee8 aakhundov [ghstack-poisoned]

ghstack-source-id: 8db39bd7bfd6113e4c1a187c092b7643934817ce Pull Request resolved: #107617

jansel

tests are failing

torch/_inductor/codegen/triton.py

Inductor kernel codegen previously have the following side effect: - in `Kernel.__exit__ `, we add local used buffers in graph.removed_buffers - during codegen, we do memory allocation/free. These cause doing multiple versions of codegen for the same kernel hard. The PR refactor the code to make kernel codegen not changing graph level states. After codegening a kernel, the graph level state is not changed so we can go on to codegen another version of the kernel if we want. cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx peterbell10 ipiszy ngimel yf225 chenyang78 kadeng muchulee8 aakhundov [ghstack-poisoned]

ghstack-source-id: 1fa6985e2f337e6617d5cc8c6574581588b03070 Pull Request resolved: #107617

Inductor kernel codegen previously have the following side effect: - in `Kernel.__exit__ `, we add local used buffers in graph.removed_buffers - during codegen, we do memory allocation/free. These cause doing multiple versions of codegen for the same kernel hard. The PR refactor the code to make kernel codegen not changing graph level states. After codegening a kernel, the graph level state is not changed so we can go on to codegen another version of the kernel if we want. cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx peterbell10 ipiszy ngimel yf225 chenyang78 kadeng muchulee8 aakhundov [ghstack-poisoned]

ghstack-source-id: 80b502ef7f0c3ab8fd969cb0f2b186cc1d250a3d Pull Request resolved: #107617

Inductor kernel codegen previously have the following side effect: - in `Kernel.__exit__ `, we add local used buffers in graph.removed_buffers - during codegen, we do memory allocation/free. These cause doing multiple versions of codegen for the same kernel hard. The PR refactor the code to make kernel codegen not changing graph level states. After codegening a kernel, the graph level state is not changed so we can go on to codegen another version of the kernel if we want. cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx peterbell10 ipiszy ngimel yf225 chenyang78 kadeng muchulee8 aakhundov [ghstack-poisoned]

ghstack-source-id: d061b532c793e4d74c5b677b8be4e87ff7e3385f Pull Request resolved: #107617

Inductor kernel codegen previously have the following side effect: - in `Kernel.__exit__ `, we add local used buffers in graph.removed_buffers - during codegen, we do memory allocation/free. These cause doing multiple versions of codegen for the same kernel hard. The PR refactor the code to make kernel codegen not changing graph level states. After codegening a kernel, the graph level state is not changed so we can go on to codegen another version of the kernel if we want. cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx peterbell10 ipiszy ngimel yf225 chenyang78 kadeng muchulee8 aakhundov [ghstack-poisoned]

ghstack-source-id: 2c2d37aed73240bbd685dd348a97db3b1fba02ea Pull Request resolved: #107617

Inductor kernel codegen previously have the following side effect: - in `Kernel.__exit__ `, we add local used buffers in graph.removed_buffers - during codegen, we do memory allocation/free. These cause doing multiple versions of codegen for the same kernel hard. The PR refactor the code to make kernel codegen not changing graph level states. After codegening a kernel, the graph level state is not changed so we can go on to codegen another version of the kernel if we want. cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx peterbell10 ipiszy ngimel yf225 chenyang78 kadeng muchulee8 aakhundov [ghstack-poisoned]

ghstack-source-id: 67b8cb60abdb509ecda45b613eb60d5e023dcd0b Pull Request resolved: #107617

Inductor kernel codegen previously have the following side effect: - in `Kernel.__exit__ `, we add local used buffers in graph.removed_buffers - during codegen, we do memory allocation/free. These cause doing multiple versions of codegen for the same kernel hard. The PR refactor the code to make kernel codegen not changing graph level states. After codegening a kernel, the graph level state is not changed so we can go on to codegen another version of the kernel if we want. cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx peterbell10 ipiszy ngimel yf225 chenyang78 kadeng muchulee8 aakhundov [ghstack-poisoned]

ghstack-source-id: 198775dfe581bd32b33082974800cb96d8a40b2f Pull Request resolved: #107617

shunting314 · 2023-08-30T05:15:09Z

@pytorchbot merge

pytorchmergebot · 2023-08-30T05:17:02Z

Merge failed

Reason: This PR needs a release notes: label
If your changes are user facing and intended to be a part of release notes, please use a label starting with release notes:.

If not, please add the topic: not user facing label.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "topic: not user facing"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

Details for Dev Infra team

Raised by workflow job

shunting314 · 2023-08-30T07:03:31Z

@pytorchbot label "topic: not user facing"

shunting314 · 2023-08-30T07:03:49Z

@pytorchbot merge

pytorchmergebot · 2023-08-30T07:05:49Z

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

pytorchmergebot · 2023-08-30T10:43:52Z

Merge failed

Reason: Command git -C /home/runner/work/pytorch/pytorch cherry-pick -x ec1824c9e44e5a0f5783bac6d3188b7de5a3bd24 returned non-zero exit code 1

Auto-merging torch/_inductor/codegen/triton.py
Auto-merging torch/_inductor/codegen/triton_utils.py
CONFLICT (content): Merge conflict in torch/_inductor/codegen/triton_utils.py
Auto-merging torch/_inductor/codegen/wrapper.py
Auto-merging torch/_inductor/graph.py
error: could not apply ec1824c9e44... [inductor] no-side-effect codegen
hint: After resolving the conflicts, mark them with
hint: "git add/rm <pathspec>", then run
hint: "git cherry-pick --continue".
hint: You can instead skip this commit with "git cherry-pick --skip".
hint: To abort and get back to the state before "git cherry-pick",
hint: run "git cherry-pick --abort".

Details for Dev Infra team

Raised by workflow job

Inductor kernel codegen previously have the following side effect: - in `Kernel.__exit__ `, we add local used buffers in graph.removed_buffers - during codegen, we do memory allocation/free. These cause doing multiple versions of codegen for the same kernel hard. The PR refactor the code to make kernel codegen not changing graph level states. After codegening a kernel, the graph level state is not changed so we can go on to codegen another version of the kernel if we want. cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx peterbell10 ipiszy ngimel yf225 chenyang78 kadeng muchulee8 aakhundov [ghstack-poisoned]

shunting314 · 2023-08-30T21:10:39Z

@pytorchbot merge

pytorchmergebot · 2023-08-30T21:12:59Z

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

[inductor] no-side-effect codegen

7fadc62

[ghstack-poisoned]

shunting314 mentioned this pull request Aug 21, 2023

[inductor] let codegen not rely on node order #107320

Closed

github-actions bot added module: inductor ciflow/inductor labels Aug 21, 2023

shunting314 added a commit that referenced this pull request Aug 21, 2023

[inductor] no-side-effect codegen

f4b0229

ghstack-source-id: 03e8e58847a16268605f7a3156dfaf3785f6d9de Pull Request resolved: #107617

shunting314 requested a review from jansel August 21, 2023 19:31

shunting314 added a commit that referenced this pull request Aug 21, 2023

[inductor] no-side-effect codegen

0579465

ghstack-source-id: 5edd38d351ba5f15d390de5d890ca6eb29215249 Pull Request resolved: #107617

shunting314 added a commit that referenced this pull request Aug 21, 2023

[inductor] no-side-effect codegen

1d94ced

ghstack-source-id: 236c6902f23f217cb6d36c8a70dc622e98a82cf2 Pull Request resolved: #107617

shunting314 added a commit that referenced this pull request Aug 22, 2023

[inductor] no-side-effect codegen

1660bfe

ghstack-source-id: a20bfba952cb862e564e984c6b1ca248ccdbc5f8 Pull Request resolved: #107617

shunting314 added a commit that referenced this pull request Aug 23, 2023

[inductor] no-side-effect codegen

1096973

ghstack-source-id: 8db39bd7bfd6113e4c1a187c092b7643934817ce Pull Request resolved: #107617

jansel requested changes Aug 23, 2023

View reviewed changes

torch/_inductor/codegen/triton.py Show resolved Hide resolved

shunting314 added a commit that referenced this pull request Aug 25, 2023

[inductor] no-side-effect codegen

b2943d3

ghstack-source-id: 1fa6985e2f337e6617d5cc8c6574581588b03070 Pull Request resolved: #107617

shunting314 added a commit that referenced this pull request Aug 26, 2023

[inductor] no-side-effect codegen

992e7d1

ghstack-source-id: 80b502ef7f0c3ab8fd969cb0f2b186cc1d250a3d Pull Request resolved: #107617

shunting314 added a commit that referenced this pull request Aug 27, 2023

[inductor] no-side-effect codegen

750c720

ghstack-source-id: d061b532c793e4d74c5b677b8be4e87ff7e3385f Pull Request resolved: #107617

shunting314 added a commit that referenced this pull request Aug 29, 2023

[inductor] no-side-effect codegen

6073bdf

ghstack-source-id: 2c2d37aed73240bbd685dd348a97db3b1fba02ea Pull Request resolved: #107617

shunting314 added a commit that referenced this pull request Aug 29, 2023

[inductor] no-side-effect codegen

973d5ef

ghstack-source-id: 67b8cb60abdb509ecda45b613eb60d5e023dcd0b Pull Request resolved: #107617

shunting314 added a commit that referenced this pull request Aug 29, 2023

[inductor] no-side-effect codegen

ec1824c

ghstack-source-id: 198775dfe581bd32b33082974800cb96d8a40b2f Pull Request resolved: #107617

shunting314 requested a review from jansel August 29, 2023 19:15

jansel approved these changes Aug 29, 2023

View reviewed changes

shunting314 mentioned this pull request Aug 29, 2023

[inductor] benchmark fusion #108193

Closed

pytorch-bot bot added the ciflow/trunk Trigger trunk jobs on your pull request label Aug 30, 2023

pytorchmergebot added the merging label Aug 30, 2023

pytorchmergebot removed the merging label Aug 30, 2023

pytorch-bot bot added the topic: not user facing topic category label Aug 30, 2023

pytorchmergebot added the merging label Aug 30, 2023

pytorchmergebot removed the merging label Aug 30, 2023

pytorchmergebot added the merging label Aug 30, 2023

pytorchmergebot added Merged and removed merging labels Aug 31, 2023

pytorchmergebot closed this in 7cb4bf6 Aug 31, 2023

facebook-github-bot deleted the gh/shunting314/73/head branch September 3, 2023 14:24

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[inductor] no-side-effect codegen #107617

[inductor] no-side-effect codegen #107617

shunting314 commented Aug 21, 2023 •

edited

pytorch-bot bot commented Aug 21, 2023 •

edited

jansel left a comment

shunting314 commented Aug 30, 2023

pytorchmergebot commented Aug 30, 2023

shunting314 commented Aug 30, 2023

shunting314 commented Aug 30, 2023

pytorchmergebot commented Aug 30, 2023

pytorchmergebot commented Aug 30, 2023

shunting314 commented Aug 30, 2023

pytorchmergebot commented Aug 30, 2023

[inductor] no-side-effect codegen #107617

[inductor] no-side-effect codegen #107617

Conversation

shunting314 commented Aug 21, 2023 • edited

pytorch-bot bot commented Aug 21, 2023 • edited

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/107617

✅ No Failures

jansel left a comment

Choose a reason for hiding this comment

shunting314 commented Aug 30, 2023

pytorchmergebot commented Aug 30, 2023

Merge failed

shunting314 commented Aug 30, 2023

shunting314 commented Aug 30, 2023

pytorchmergebot commented Aug 30, 2023

Merge started

pytorchmergebot commented Aug 30, 2023

Merge failed

shunting314 commented Aug 30, 2023

pytorchmergebot commented Aug 30, 2023

Merge started

shunting314 commented Aug 21, 2023 •

edited

pytorch-bot bot commented Aug 21, 2023 •

edited