fix torch.compile with triton kernels under inference_mode #124489

bdhirsh · 2024-04-19T15:25:03Z

Stack from ghstack (oldest at bottom):

-> fix torch.compile with triton kernels under inference_mode #124489

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @yf225 @chenyang78 @kadeng @muchulee8 @aakhundov @ColinPeppler @amjames @desertfire @chauhang

[ghstack-poisoned]

pytorch-bot · 2024-04-19T15:25:07Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/124489

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (2 Unrelated Failures)

As of commit 1b38515 with merge base 01694ea ():

UNSTABLE - The following jobs failed but were likely due to flakiness present on trunk and has been marked as unstable:

inductor / cuda12.1-py3.10-gcc9-sm86 / test (dynamic_inductor_timm, 2, 2, linux.g5.4xlarge.nvidia.gpu, unstable) (gh) ()
sebotnet33ts_256
inductor / linux-jammy-cpu-py3.8-gcc11-inductor / test (inductor_torchbench_cpu_smoketest_perf, 1, 1, linux.24xl.spr-metal, unstable) (gh) (#126993)
Process completed with exit code 1.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

ghstack-source-id: 6be489035b2dfe6491e9fef5457d5a54b779aefb Pull Request resolved: #124489

albanD

SGTM

cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx peterbell10 ipiszy yf225 chenyang78 kadeng muchulee8 aakhundov ColinPeppler amjames desertfire chauhang [ghstack-poisoned]

ghstack-source-id: 4ec86147e5fabf057599ce83b54228316b4bc92a Pull Request resolved: #124489

cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx peterbell10 ipiszy yf225 chenyang78 kadeng muchulee8 aakhundov ColinPeppler amjames desertfire chauhang [ghstack-poisoned]

ghstack-source-id: 8cd261591bc63711ac14d3870b88cd18e1a754e9 Pull Request resolved: #124489

cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx peterbell10 ipiszy yf225 chenyang78 kadeng muchulee8 aakhundov ColinPeppler amjames desertfire chauhang [ghstack-poisoned]

ghstack-source-id: 5a06e52d6604320f2b554fcbce2d9be95ca886c9 Pull Request resolved: #124489

cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx peterbell10 ipiszy yf225 chenyang78 kadeng muchulee8 aakhundov ColinPeppler amjames desertfire chauhang [ghstack-poisoned]

ghstack-source-id: 10506fba19ef845c4add8eb631372a89d10a5878 Pull Request resolved: #124489

cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx peterbell10 ipiszy yf225 chenyang78 kadeng muchulee8 aakhundov ColinPeppler amjames desertfire chauhang [ghstack-poisoned]

ghstack-source-id: 70c93e98a1036b0acbb602cd2a2c7469d73e2ec1 Pull Request resolved: #124489

cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx peterbell10 ipiszy yf225 chenyang78 kadeng muchulee8 aakhundov ColinPeppler amjames desertfire chauhang [ghstack-poisoned]

ghstack-source-id: 5cb5e629a9e78473a153aac4f46112cf7af7544e Pull Request resolved: #124489

cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx peterbell10 ipiszy yf225 chenyang78 kadeng muchulee8 aakhundov ColinPeppler amjames desertfire chauhang [ghstack-poisoned]

ghstack-source-id: 9dbe738344ffc9b0dd53c7456016b983f0def550 Pull Request resolved: #124489

cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx peterbell10 ipiszy yf225 chenyang78 kadeng muchulee8 aakhundov ColinPeppler amjames desertfire chauhang [ghstack-poisoned]

ghstack-source-id: 156179177202099787e7a113be3922aa454ce7a0 Pull Request resolved: #124489

cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx peterbell10 ipiszy yf225 chenyang78 kadeng muchulee8 aakhundov ColinPeppler amjames desertfire chauhang [ghstack-poisoned]

ghstack-source-id: c21abeee274590938bd1086477a7d3f1ae84f2b7 Pull Request resolved: #124489

bdhirsh · 2024-06-05T21:44:30Z

@pytorchbot merge

pytorchmergebot · 2024-06-05T21:46:24Z

Merge failed

Reason: This PR needs a release notes: label
If your changes are user facing and intended to be a part of release notes, please use a label starting with release notes:.

If not, please add the topic: not user facing label.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "topic: not user facing"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

Details for Dev Infra team

Raised by workflow job

cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx peterbell10 ipiszy yf225 chenyang78 kadeng muchulee8 aakhundov ColinPeppler amjames desertfire chauhang [ghstack-poisoned]

ghstack-source-id: 59d5f6121ac996751b9ecc4ff7e4e75d922dd439 Pull Request resolved: #124489

bdhirsh · 2024-06-06T03:10:13Z

@pytorchbot merge

pytorchmergebot · 2024-06-06T03:11:57Z

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

pytorchmergebot · 2024-06-06T09:10:42Z

The merge job was canceled or timed out. This most often happen if two merge requests were issued for the same PR, or if merge job was waiting for more than 6 hours for tests to finish. In later case, please do not hesitate to reissue the merge command
For more information see pytorch-bot wiki.

bdhirsh · 2024-06-06T17:01:41Z

@pytorchbot merge

pytorchmergebot · 2024-06-06T17:03:47Z

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

pytorchmergebot · 2024-06-06T23:02:11Z

The merge job was canceled or timed out. This most often happen if two merge requests were issued for the same PR, or if merge job was waiting for more than 6 hours for tests to finish. In later case, please do not hesitate to reissue the merge command
For more information see pytorch-bot wiki.

oulgen · 2024-06-07T01:07:11Z

@pytorchbot merge

pytorchmergebot · 2024-06-07T01:08:51Z

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

…24489) Pull Request resolved: pytorch#124489 Approved by: https://github.com/albanD

fix torch.compile with triton kernels under inference_mode

87bc894

[ghstack-poisoned]

bdhirsh mentioned this pull request Apr 19, 2024

AOTAutograd: gate view-replay behind config, not the default #124488

Closed

pytorch-bot bot added ciflow/inductor module: inductor labels Apr 19, 2024

bdhirsh added a commit that referenced this pull request Apr 19, 2024

fix torch.compile with triton kernels under inference_mode

231856b

ghstack-source-id: 6be489035b2dfe6491e9fef5457d5a54b779aefb Pull Request resolved: #124489

github-actions bot requested review from albanD, antoniojkim, ezyang, miladm and SherlockNoMad April 19, 2024 15:25

albanD approved these changes Apr 19, 2024

View reviewed changes

bdhirsh added a commit that referenced this pull request Apr 22, 2024

fix torch.compile with triton kernels under inference_mode

ca1ce02

ghstack-source-id: 4ec86147e5fabf057599ce83b54228316b4bc92a Pull Request resolved: #124489

This was referenced Apr 24, 2024

update xla pin #124839

Open

Reland "AOTAutograd: gate view-replay behind config, not the default (#124488)"" #124840

Closed

bdhirsh added a commit that referenced this pull request Apr 24, 2024

fix torch.compile with triton kernels under inference_mode

d37a82c

ghstack-source-id: 8cd261591bc63711ac14d3870b88cd18e1a754e9 Pull Request resolved: #124489

bdhirsh added a commit that referenced this pull request Apr 25, 2024

fix torch.compile with triton kernels under inference_mode

d16cc04

ghstack-source-id: 5a06e52d6604320f2b554fcbce2d9be95ca886c9 Pull Request resolved: #124489

bdhirsh added a commit that referenced this pull request Apr 25, 2024

fix torch.compile with triton kernels under inference_mode

a2718a3

ghstack-source-id: 10506fba19ef845c4add8eb631372a89d10a5878 Pull Request resolved: #124489

bdhirsh added a commit that referenced this pull request Apr 25, 2024

fix torch.compile with triton kernels under inference_mode

f9b3cf2

ghstack-source-id: 70c93e98a1036b0acbb602cd2a2c7469d73e2ec1 Pull Request resolved: #124489

bdhirsh added a commit that referenced this pull request Apr 25, 2024

fix torch.compile with triton kernels under inference_mode

56e41d6

ghstack-source-id: 5cb5e629a9e78473a153aac4f46112cf7af7544e Pull Request resolved: #124489

bdhirsh added a commit that referenced this pull request Apr 26, 2024

fix torch.compile with triton kernels under inference_mode

e55d06a

ghstack-source-id: 9dbe738344ffc9b0dd53c7456016b983f0def550 Pull Request resolved: #124489

bdhirsh mentioned this pull request May 14, 2024

torch.compile uses customed trition kernel reports: RuntimeError: Inference tensors do not track version counter #125989

Closed

bdhirsh added a commit that referenced this pull request May 15, 2024

fix torch.compile with triton kernels under inference_mode

3486900

ghstack-source-id: 156179177202099787e7a113be3922aa454ce7a0 Pull Request resolved: #124489

bdhirsh added a commit that referenced this pull request Jun 5, 2024

fix torch.compile with triton kernels under inference_mode

5beddfa

ghstack-source-id: c21abeee274590938bd1086477a7d3f1ae84f2b7 Pull Request resolved: #124489

pytorch-bot bot added the ciflow/trunk Trigger trunk jobs on your pull request label Jun 5, 2024

pytorchmergebot added the merging label Jun 5, 2024

pytorchmergebot removed the merging label Jun 5, 2024

bdhirsh added the release notes: composability release notes category label Jun 6, 2024

bdhirsh added a commit that referenced this pull request Jun 6, 2024

fix torch.compile with triton kernels under inference_mode

53a7663

ghstack-source-id: 59d5f6121ac996751b9ecc4ff7e4e75d922dd439 Pull Request resolved: #124489

pytorchmergebot added the merging label Jun 6, 2024

pytorchmergebot closed this in 476bfe6 Jun 7, 2024

pytorchmergebot added Merged and removed merging labels Jun 7, 2024

This was referenced Jun 7, 2024

torch.compile doesn't work well with custom triton kernel from Mamba #128061

Closed

torch.compile-ing triton kernels with inputs created under inference mode leads to error #124111

Closed

TharinduRusira pushed a commit to TharinduRusira/pytorch that referenced this pull request Jun 14, 2024

fix torch.compile with triton kernels under inference_mode (pytorch#1…

706c435

…24489) Pull Request resolved: pytorch#124489 Approved by: https://github.com/albanD

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix torch.compile with triton kernels under inference_mode #124489

fix torch.compile with triton kernels under inference_mode #124489

bdhirsh commented Apr 19, 2024 •

edited

Loading

pytorch-bot bot commented Apr 19, 2024 •

edited

Loading

albanD left a comment

bdhirsh commented Jun 5, 2024

pytorchmergebot commented Jun 5, 2024

bdhirsh commented Jun 6, 2024

pytorchmergebot commented Jun 6, 2024

pytorchmergebot commented Jun 6, 2024

bdhirsh commented Jun 6, 2024

pytorchmergebot commented Jun 6, 2024

pytorchmergebot commented Jun 6, 2024

oulgen commented Jun 7, 2024

pytorchmergebot commented Jun 7, 2024

fix torch.compile with triton kernels under inference_mode #124489

fix torch.compile with triton kernels under inference_mode #124489

Conversation

bdhirsh commented Apr 19, 2024 • edited Loading

pytorch-bot bot commented Apr 19, 2024 • edited Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/124489

✅ You can merge normally! (2 Unrelated Failures)

albanD left a comment

Choose a reason for hiding this comment

bdhirsh commented Jun 5, 2024

pytorchmergebot commented Jun 5, 2024

Merge failed

bdhirsh commented Jun 6, 2024

pytorchmergebot commented Jun 6, 2024

Merge started

pytorchmergebot commented Jun 6, 2024

bdhirsh commented Jun 6, 2024

pytorchmergebot commented Jun 6, 2024

Merge started

pytorchmergebot commented Jun 6, 2024

oulgen commented Jun 7, 2024

pytorchmergebot commented Jun 7, 2024

Merge started

bdhirsh commented Apr 19, 2024 •

edited

Loading

pytorch-bot bot commented Apr 19, 2024 •

edited

Loading