Compute bounds for the variables created during codegen #123100

lezcano · 2024-04-01T15:56:13Z

Stack from ghstack (oldest at bottom):

-> Compute bounds for the variables created during codegen #123100

Before we would just bail out on these bounds for all variables that did
not come from the FX graph. Now we propagate the bounds whenever we have
a rule for that op.

cc @jgong5 @mingfeima @XiaobingSuper @sanchitintel @ashokei @jingxu10 @gujinghui @PenghuiCheng @jianyuh @min-jean-cho @yanbing-j @Guobing-Chen @Xia-Weiwen @snadampal @voznesenskym @penguinwu @EikanWang @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @yf225 @chenyang78 @kadeng @muchulee8 @ColinPeppler @amjames @desertfire @chauhang @aakhundov

Before we would just bail out on these bounds for all variables that did not come from the FX graph. Now we propagate the bounds whenever we have a rule for that op. [ghstack-poisoned]

pytorch-bot · 2024-04-01T15:56:16Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/123100

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (1 Unrelated Failure)

As of commit 30ec168 with merge base 7ddf57e ():

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

inductor / rocm6.1-py3.8-inductor / test (inductor, 1, 1, linux.rocm.gpu.2) (gh) (trunk failure)
test/distributed/_composable/fsdp/test_fully_shard_training.py::TestFullyShard2DTraining::test_train_parity_2d_mlp

This comment was automatically generated by Dr. CI and updates every 15 minutes.

Before we would just bail out on these bounds for all variables that did not come from the FX graph. Now we propagate the bounds whenever we have a rule for that op. ghstack-source-id: da22cfa72245b7608da154079399039c57ea9192 Pull Request resolved: #123100

Before we would just bail out on these bounds for all variables that did not come from the FX graph. Now we propagate the bounds whenever we have a rule for that op. cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx peterbell10 ipiszy yf225 chenyang78 kadeng muchulee8 aakhundov ColinPeppler amjames desertfire chauhang [ghstack-poisoned]

Before we would just bail out on these bounds for all variables that did not come from the FX graph. Now we propagate the bounds whenever we have a rule for that op. ghstack-source-id: 72e24d911dee58b743430313e191c49b1599fb98 Pull Request resolved: #123100

Before we would just bail out on these bounds for all variables that did not come from the FX graph. Now we propagate the bounds whenever we have a rule for that op. cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx peterbell10 ipiszy yf225 chenyang78 kadeng muchulee8 aakhundov ColinPeppler amjames desertfire chauhang [ghstack-poisoned]

Before we would just bail out on these bounds for all variables that did not come from the FX graph. Now we propagate the bounds whenever we have a rule for that op. ghstack-source-id: 33c1f822455ac6d0318d24f2c54602696a86bbf1 Pull Request resolved: #123100

Before we would just bail out on these bounds for all variables that did not come from the FX graph. Now we propagate the bounds whenever we have a rule for that op. cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx peterbell10 ipiszy yf225 chenyang78 kadeng muchulee8 aakhundov ColinPeppler amjames desertfire chauhang [ghstack-poisoned]

Before we would just bail out on these bounds for all variables that did not come from the FX graph. Now we propagate the bounds whenever we have a rule for that op. ghstack-source-id: 1c876e3ca231060d27e54d2847bfc2316ce4c405 Pull Request resolved: #123100

Before we would just bail out on these bounds for all variables that did not come from the FX graph. Now we propagate the bounds whenever we have a rule for that op. cc jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 voznesenskym penguinwu EikanWang Guobing-Chen zhuhaozhe blzheng wenzhe-nrv jiayisunx peterbell10 ipiszy yf225 chenyang78 kadeng muchulee8 aakhundov ColinPeppler amjames desertfire chauhang [ghstack-poisoned]

Before we would just bail out on these bounds for all variables that did not come from the FX graph. Now we propagate the bounds whenever we have a rule for that op. ghstack-source-id: a9933014d1163b442cf688dff366f65e37827db3 Pull Request resolved: #123100

Before we would just bail out on these bounds for all variables that did not come from the FX graph. Now we propagate the bounds whenever we have a rule for that op. cc jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 voznesenskym penguinwu EikanWang Guobing-Chen zhuhaozhe blzheng wenzhe-nrv jiayisunx peterbell10 ipiszy yf225 chenyang78 kadeng muchulee8 aakhundov ColinPeppler amjames desertfire chauhang [ghstack-poisoned]

Before we would just bail out on these bounds for all variables that did not come from the FX graph. Now we propagate the bounds whenever we have a rule for that op. ghstack-source-id: ef39f694834ab830f8b845f46ef80a1755968552 Pull Request resolved: #123100

Before we would just bail out on these bounds for all variables that did not come from the FX graph. Now we propagate the bounds whenever we have a rule for that op. cc jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 voznesenskym penguinwu EikanWang Guobing-Chen zhuhaozhe blzheng wenzhe-nrv jiayisunx peterbell10 ipiszy yf225 chenyang78 kadeng muchulee8 aakhundov ColinPeppler amjames desertfire chauhang [ghstack-poisoned]

Before we would just bail out on these bounds for all variables that did not come from the FX graph. Now we propagate the bounds whenever we have a rule for that op. ghstack-source-id: a5721cff4ee9c89fe76248401c480f7efa38f2dc Pull Request resolved: #123100

peterbell10 · 2024-04-02T15:28:30Z

torch/_inductor/codegen/common.py

+                    # We will also likely not get much from computing VRs on these nodes
+                    if any(
+                        s in fx_node.target
+                        for s in ("set_indirect", "reduction", "scan")


Not necessary for now, but some reductions and/or scans should be really easy. e.g. min/max doesn't change the bounds at all.

We should probably fix that in the pass that computes the bounds in the FX graph, not here, but yep.

torch/utils/_sympy/value_ranges.py

torch/_inductor/codegen/triton.py

torch/_inductor/codegen/common.py

Before we would just bail out on these bounds for all variables that did not come from the FX graph. Now we propagate the bounds whenever we have a rule for that op. cc jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 voznesenskym penguinwu EikanWang Guobing-Chen zhuhaozhe blzheng wenzhe-nrv jiayisunx peterbell10 ipiszy yf225 chenyang78 kadeng muchulee8 aakhundov ColinPeppler amjames desertfire chauhang [ghstack-poisoned]

Before we would just bail out on these bounds for all variables that did not come from the FX graph. Now we propagate the bounds whenever we have a rule for that op. ghstack-source-id: 96f095cce7b92c0409d20b0fd0ab5f88e9f1ac4a Pull Request resolved: #123100

Before we would just bail out on these bounds for all variables that did not come from the FX graph. Now we propagate the bounds whenever we have a rule for that op. cc jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 voznesenskym penguinwu EikanWang Guobing-Chen zhuhaozhe blzheng wenzhe-nrv jiayisunx peterbell10 ipiszy yf225 chenyang78 kadeng muchulee8 aakhundov ColinPeppler amjames desertfire chauhang [ghstack-poisoned]

Before we would just bail out on these bounds for all variables that did not come from the FX graph. Now we propagate the bounds whenever we have a rule for that op. ghstack-source-id: 7147b801f9f766a30972e3cb1cad2b5cfb0c122d Pull Request resolved: #123100

Before we would just bail out on these bounds for all variables that did not come from the FX graph. Now we propagate the bounds whenever we have a rule for that op. cc jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 voznesenskym penguinwu EikanWang Guobing-Chen zhuhaozhe blzheng wenzhe-nrv jiayisunx peterbell10 ipiszy yf225 chenyang78 kadeng muchulee8 aakhundov ColinPeppler amjames desertfire chauhang [ghstack-poisoned]

Before we would just bail out on these bounds for all variables that did not come from the FX graph. Now we propagate the bounds whenever we have a rule for that op. ghstack-source-id: e3d3673d58caccc4ec86813576b0534db12d2284 Pull Request resolved: #123100

torch/_inductor/codegen/cpp.py

huydhn · 2024-05-06T20:20:41Z

@pytorchbot revert -m 'Sorry for reverting you change but it is failing inductor tests https://hud.pytorch.org/pytorch/pytorch/commit/bb668c6468dd4adf7737a069e7af4c3f612cfc81' -c landrace

pytorchmergebot · 2024-05-06T20:23:31Z

@pytorchbot successfully started a revert job. Check the current status here.
Questions? Feedback? Please reach out to the PyTorch DevX Team

)" This reverts commit bb668c6. Reverted #123100 on behalf of https://github.com/huydhn due to Sorry for reverting you change but it is failing inductor tests https://hud.pytorch.org/pytorch/pytorch/commit/bb668c6468dd4adf7737a069e7af4c3f612cfc81 ([comment](#123100 (comment)))

pytorchmergebot · 2024-05-06T20:23:43Z

@lezcano your PR has been successfully reverted.

Before we would just bail out on these bounds for all variables that did not come from the FX graph. Now we propagate the bounds whenever we have a rule for that op. cc jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 voznesenskym penguinwu EikanWang Guobing-Chen zhuhaozhe blzheng wenzhe-nrv jiayisunx peterbell10 ipiszy yf225 chenyang78 kadeng muchulee8 aakhundov ColinPeppler amjames desertfire chauhang [ghstack-poisoned]

Before we would just bail out on these bounds for all variables that did not come from the FX graph. Now we propagate the bounds whenever we have a rule for that op. ghstack-source-id: 5cc1df9ede3db690bfa67a1e604e07b811e77e2f Pull Request resolved: #123100

Before we would just bail out on these bounds for all variables that did not come from the FX graph. Now we propagate the bounds whenever we have a rule for that op. cc jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 voznesenskym penguinwu EikanWang Guobing-Chen zhuhaozhe blzheng wenzhe-nrv jiayisunx peterbell10 ipiszy yf225 chenyang78 kadeng muchulee8 aakhundov ColinPeppler amjames desertfire chauhang [ghstack-poisoned]

Before we would just bail out on these bounds for all variables that did not come from the FX graph. Now we propagate the bounds whenever we have a rule for that op. ghstack-source-id: aed83e4ea7eed7e06274dcca355f6ae84e1d1444 Pull Request resolved: #123100

lezcano · 2024-05-07T17:13:05Z

@pytorchbot merge

pytorchmergebot · 2024-05-07T17:15:27Z

This PR updates submodules third_party/ideep

If those updates are intentional, please add "submodule" keyword to PR title/description.

Before we would just bail out on these bounds for all variables that did not come from the FX graph. Now we propagate the bounds whenever we have a rule for that op. cc jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 gujinghui PenghuiCheng jianyuh min-jean-cho yanbing-j Guobing-Chen Xia-Weiwen snadampal voznesenskym penguinwu EikanWang zhuhaozhe blzheng wenzhe-nrv jiayisunx peterbell10 ipiszy yf225 chenyang78 kadeng muchulee8 ColinPeppler amjames desertfire chauhang aakhundov [ghstack-poisoned]

lezcano · 2024-05-07T18:39:36Z

@pytorchbot merge

Before we would just bail out on these bounds for all variables that did not come from the FX graph. Now we propagate the bounds whenever we have a rule for that op. ghstack-source-id: 55e5fdf2b2bd5d357379476a837230a9618ea033 Pull Request resolved: #123100

pytorchmergebot · 2024-05-07T18:42:32Z

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

pytorchmergebot · 2024-05-08T00:41:03Z

The merge job was canceled or timed out. This most often happen if two merge requests were issued for the same PR, or if merge job was waiting for more than 6 hours for tests to finish. In later case, please do not hesitate to reissue the merge command
For more information see pytorch-bot wiki.

lezcano · 2024-05-08T08:11:48Z

@pytorchbot merge

pytorchmergebot · 2024-05-08T08:13:41Z

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

Compute bounds for the variables created during codegen

0e09975

Before we would just bail out on these bounds for all variables that did not come from the FX graph. Now we propagate the bounds whenever we have a rule for that op. [ghstack-poisoned]

lezcano mentioned this pull request Apr 1, 2024

Do not forward parent's value range to CSE variable for variables created within codegen. #123099

Closed

pytorch-bot bot added ciflow/inductor module: inductor labels Apr 1, 2024

pytorchbot added the open source label Apr 1, 2024

pytorch-bot bot added the module: cpu CPU specific problem (e.g., perf, algorithm) label Apr 2, 2024

lezcano requested review from peterbell10 and jgong5 April 2, 2024 14:17

peterbell10 reviewed Apr 2, 2024

View reviewed changes

jgong5 reviewed Apr 3, 2024

View reviewed changes

torch/_inductor/codegen/cpp.py Outdated Show resolved Hide resolved

pytorchmergebot closed this in bb668c6 May 6, 2024

pytorchmergebot removed the merging label May 6, 2024

pytorchmergebot added the Reverted label May 6, 2024

pytorchmergebot reopened this May 6, 2024

pytorch-bot bot added ciflow/linux-aarch64 linux aarch64 CI workflow module: mkldnn Related to Intel IDEEP or oneDNN (a.k.a. mkldnn) integration labels May 7, 2024

pytorchmergebot added the merging label May 7, 2024

pytorchmergebot closed this in 320af5e May 8, 2024

pytorchmergebot removed the merging label May 8, 2024

lezcano mentioned this pull request May 14, 2024

Fix mask propagation in the presence of where #125574

Closed

github-actions bot deleted the gh/Lezcano/291/head branch June 8, 2024 01:54

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Compute bounds for the variables created during codegen #123100

Compute bounds for the variables created during codegen #123100

lezcano commented Apr 1, 2024 •

edited by pytorch-bot bot

Loading

pytorch-bot bot commented Apr 1, 2024 •

edited

Loading

peterbell10 Apr 2, 2024

lezcano Apr 3, 2024

huydhn commented May 6, 2024

pytorchmergebot commented May 6, 2024

pytorchmergebot commented May 6, 2024

lezcano commented May 7, 2024

pytorchmergebot commented May 7, 2024

lezcano commented May 7, 2024

pytorchmergebot commented May 7, 2024

pytorchmergebot commented May 8, 2024

lezcano commented May 8, 2024

pytorchmergebot commented May 8, 2024

Compute bounds for the variables created during codegen #123100

Compute bounds for the variables created during codegen #123100

Conversation

lezcano commented Apr 1, 2024 • edited by pytorch-bot bot Loading

pytorch-bot bot commented Apr 1, 2024 • edited Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/123100

✅ You can merge normally! (1 Unrelated Failure)

peterbell10 Apr 2, 2024

Choose a reason for hiding this comment

lezcano Apr 3, 2024

Choose a reason for hiding this comment

huydhn commented May 6, 2024

pytorchmergebot commented May 6, 2024

pytorchmergebot commented May 6, 2024

lezcano commented May 7, 2024

pytorchmergebot commented May 7, 2024

lezcano commented May 7, 2024

pytorchmergebot commented May 7, 2024

Merge started

pytorchmergebot commented May 8, 2024

lezcano commented May 8, 2024

pytorchmergebot commented May 8, 2024

Merge started

lezcano commented Apr 1, 2024 •

edited by pytorch-bot bot

Loading

pytorch-bot bot commented Apr 1, 2024 •

edited

Loading