[inductor] modify the output_stride of ConcatKernel #122761

jiayisunx · 2024-03-27T03:57:00Z

Stack from ghstack (oldest at bottom):

-> [inductor] modify the output_stride of ConcatKernel #122761

Fix #121613.
Modify the output_stride of ConcatKernel: If any input to Concat is Pointwise, check the layout of all inputs to Pointwise, if any of the inputs is in channels_last format, set channels_last strides for the output_stride.

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @peterbell10 @ipiszy @yf225 @chenyang78 @kadeng @muchulee8 @aakhundov @ColinPeppler @amjames @desertfire @chauhang

pytorch-bot · 2024-03-27T03:57:02Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/122761

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (1 Unrelated Failure)

As of commit 0877fe9 with merge base 19f5033 ():

UNSTABLE - The following job failed but was likely due to flakiness present on trunk and has been marked as unstable:

inductor / rocm6.0-py3.8-inductor / test (inductor, 1, 1, linux.rocm.gpu.2, unstable) (gh)
test/distributed/test_c10d_functional_native.py::TestWithNCCL::test_unwaited

This comment was automatically generated by Dr. CI and updates every 15 minutes.

ghstack-source-id: 2ab4934 Pull Request resolved: #122761

[ghstack-poisoned]

ghstack-source-id: ae1b80d Pull Request resolved: #122761

Fix #121613. Modify the `output_stride` of `ConcatKernel`: If any input to `Concat` is `Pointwise`, check the layout of all inputs to `Pointwise`, if any of the inputs is in channels_last format, set channels_last strides for the `output_stride`. cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv peterbell10 ipiszy yf225 chenyang78 kadeng muchulee8 aakhundov ColinPeppler amjames desertfire chauhang [ghstack-poisoned]

ghstack-source-id: f9a41cf Pull Request resolved: #122761

Fix #121613. Modify the `output_stride` of `ConcatKernel`: If any input to `Concat` is `Pointwise`, check the layout of all inputs to `Pointwise`, if any of the inputs is in channels_last format, set channels_last strides for the `output_stride`. cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv peterbell10 ipiszy yf225 chenyang78 kadeng muchulee8 aakhundov ColinPeppler amjames desertfire chauhang [ghstack-poisoned]

leslie-fang-intel · 2024-04-03T04:46:00Z

torch/_inductor/ir.py


        output_stride = FlexibleLayout.contiguous_strides(new_size)
        # If any of the inputs is in CL format, use CL format for the output
+        any_input_is_storage_and_layout = False


any_input_is_storage_and_layout = any(is_storage_and_layout(x) for x in inputs) looks clearer?

leslie-fang-intel · 2024-04-03T04:52:10Z

torch/_inductor/ir.py

+    if isinstance(x, TensorBox):
+        return is_pointwise_with_channels_last_inputs(x.data)
+    if isinstance(x, StorageBox) and isinstance(x.data, Pointwise):
+        all_reads = x.data.get_reads()


Similar as in #122760, we can get the buffer from read.name. Will it be easy to check the buffer's layout?

Do I understand correctly that we can share some logic from #122760 too? Both need to get layouts of the input buffers.

jgong5 · 2024-04-01T14:41:13Z

torch/_inductor/ir.py

+        if any_input_is_storage_and_layout is False:
+            if all(
+                is_pointwise_with_channels_last_inputs(input) for input in inputs
+            ) and len(new_size) in [4, 5]:


is_pointwise_with_channels_last_inputs should already cover this check? Why check it again here?

jgong5 · 2024-04-03T13:07:57Z

torch/_inductor/ir.py

+                return False
+
+        return len(all_reads) >= 1 and all(
+            type(read) is dependencies.MemoryDep and is_channel_last_index(read)


Suggested change

type(read) is dependencies.MemoryDep and is_channel_last_index(read)

isinstance(read, dependencies.MemoryDep) and is_channels_last_index(read)

jgong5 · 2024-04-03T13:43:56Z

torch/_inductor/ir.py

                    output_stride = make_channels_last_strides_for(new_size)
                    break
+        if any_input_is_storage_and_layout is False:
+            if all(


explain why we requires "all" if none of the inputs is storage_and_layout instead of "any" on any input is storage_and_layout.

jgong5 · 2024-04-03T14:48:14Z

torch/_inductor/ir.py

+    if isinstance(x, TensorBox):
+        return is_pointwise_with_channels_last_inputs(x.data)
+    if isinstance(x, StorageBox) and isinstance(x.data, Pointwise):
+        all_reads = x.data.get_reads()


Do I understand correctly that we can share some logic from #122760 too? Both need to get layouts of the input buffers.

ghstack-source-id: 489aadd Pull Request resolved: #122761

Fix #121613. Modify the `output_stride` of `ConcatKernel`: If any input to `Concat` is `Pointwise`, check the layout of all inputs to `Pointwise`, if any of the inputs is in channels_last format, set channels_last strides for the `output_stride`. cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv peterbell10 ipiszy yf225 chenyang78 kadeng muchulee8 aakhundov ColinPeppler amjames desertfire chauhang [ghstack-poisoned]

ghstack-source-id: ba12c0e Pull Request resolved: #122761

Fix #121613. Modify the `output_stride` of `ConcatKernel`: If any input to `Concat` is `Pointwise`, check the layout of all inputs to `Pointwise`, if any of the inputs is in channels_last format, set channels_last strides for the `output_stride`. cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv peterbell10 ipiszy yf225 chenyang78 kadeng muchulee8 aakhundov ColinPeppler amjames desertfire chauhang [ghstack-poisoned]

ghstack-source-id: 505425f Pull Request resolved: #122761

leslie-fang-intel · 2024-04-10T11:44:38Z

LGTM, just a few more comments.

jgong5 · 2024-04-10T15:15:45Z

torch/_inductor/ir.py

+            fx_node_args = [
+                fx_node_args,
+            ]
+        if any_input_is_storage_and_layout is False and any(


suggest to add a note here explaining the heuristics we use here.

Added, thanks!

Fix #121613. Modify the `output_stride` of `ConcatKernel`: If any input to `Concat` is `Pointwise`, check the layout of all inputs to `Pointwise`, if any of the inputs is in channels_last format, set channels_last strides for the `output_stride`. cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv peterbell10 ipiszy yf225 chenyang78 kadeng muchulee8 aakhundov ColinPeppler amjames desertfire chauhang [ghstack-poisoned]

ghstack-source-id: 525e8ef Pull Request resolved: #122761

Fix #121613. Modify the `output_stride` of `ConcatKernel`: If any input to `Concat` is `Pointwise`, check the layout of all inputs to `Pointwise`, if any of the inputs is in channels_last format, set channels_last strides for the `output_stride`. cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv peterbell10 ipiszy yf225 chenyang78 kadeng muchulee8 aakhundov ColinPeppler amjames desertfire chauhang [ghstack-poisoned]

jiayisunx · 2024-04-17T04:06:46Z

@jansel , could you please review this PR? Thanks.

jiayisunx · 2024-04-18T02:12:46Z

@pytorchbot merge

pytorchmergebot · 2024-04-18T02:14:29Z

Merge failed

Reason: This PR needs a release notes: label
If your changes are user facing and intended to be a part of release notes, please use a label starting with release notes:.

If not, please add the topic: not user facing label.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "topic: not user facing"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

Details for Dev Infra team

Raised by workflow job

jiayisunx · 2024-04-18T02:18:00Z

@pytorchbot merge

pytorchmergebot · 2024-04-18T02:19:34Z

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

Fix pytorch#121613. Modify the `output_stride` of `ConcatKernel`: If any input to `Concat` is `Pointwise`, check the layout of all inputs to `Pointwise`, if any of the inputs is in channels_last format, set channels_last strides for the `output_stride`. Pull Request resolved: pytorch#122761 Approved by: https://github.com/jgong5, https://github.com/leslie-fang-intel, https://github.com/jansel

pytorch-bot bot added ciflow/inductor module: inductor labels Mar 27, 2024

jiayisunx added a commit that referenced this pull request Mar 27, 2024

[inductor] modify the output_stride of ConcatKernel

59576d5

ghstack-source-id: 2ab4934 Pull Request resolved: #122761

jiayisunx marked this pull request as draft March 27, 2024 03:57

[inductor] modify the output_stride of ConcatKernel

740e26c

[ghstack-poisoned]

pytorchbot added the open source label Mar 27, 2024

jiayisunx added a commit that referenced this pull request Mar 29, 2024

[inductor] modify the output_stride of ConcatKernel

109acb2

ghstack-source-id: ae1b80d Pull Request resolved: #122761

jiayisunx added a commit that referenced this pull request Apr 1, 2024

[inductor] modify the output_stride of ConcatKernel

166aff2

ghstack-source-id: f9a41cf Pull Request resolved: #122761

jiayisunx marked this pull request as ready for review April 1, 2024 08:21

jiayisunx requested review from jgong5 and leslie-fang-intel April 1, 2024 08:21

jiayisunx added the ciflow/trunk Trigger trunk jobs on your pull request label Apr 1, 2024

leslie-fang-intel reviewed Apr 3, 2024

View reviewed changes

jgong5 requested changes Apr 3, 2024

View reviewed changes

jiayisunx added a commit that referenced this pull request Apr 7, 2024

[inductor] modify the output_stride of ConcatKernel

aa8631a

ghstack-source-id: 489aadd Pull Request resolved: #122761

jiayisunx added a commit that referenced this pull request Apr 10, 2024

[inductor] modify the output_stride of ConcatKernel

d68cf5e

ghstack-source-id: ba12c0e Pull Request resolved: #122761

jiayisunx added a commit that referenced this pull request Apr 10, 2024

[inductor] modify the output_stride of ConcatKernel

2b15f19

ghstack-source-id: 505425f Pull Request resolved: #122761

jgong5 approved these changes Apr 10, 2024

View reviewed changes

jiayisunx added a commit that referenced this pull request Apr 15, 2024

[inductor] modify the output_stride of ConcatKernel

f1e8ef5

ghstack-source-id: 525e8ef Pull Request resolved: #122761

jiayisunx requested review from jgong5 and leslie-fang-intel April 16, 2024 05:59

leslie-fang-intel approved these changes Apr 17, 2024

View reviewed changes

jiayisunx requested a review from jansel April 17, 2024 01:26

jansel approved these changes Apr 17, 2024

View reviewed changes

pytorchmergebot added the merging label Apr 18, 2024

pytorchmergebot removed the merging label Apr 18, 2024

jiayisunx added the topic: not user facing topic category label Apr 18, 2024

pytorchmergebot added the merging label Apr 18, 2024

pytorchmergebot added the Merged label Apr 18, 2024

pytorchmergebot closed this in 51a56ef Apr 18, 2024

pytorchmergebot removed the merging label Apr 18, 2024

github-actions bot deleted the gh/jiayisunx/5/head branch May 31, 2024 01:55

	type(read) is dependencies.MemoryDep and is_channel_last_index(read)
	isinstance(read, dependencies.MemoryDep) and is_channels_last_index(read)

[inductor] modify the output_stride of ConcatKernel #122761

[inductor] modify the output_stride of ConcatKernel #122761

Uh oh!

Conversation

jiayisunx commented Mar 27, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot bot commented Mar 27, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/122761

✅ You can merge normally! (1 Unrelated Failure)

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

leslie-fang-intel commented Apr 10, 2024

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jiayisunx commented Apr 17, 2024

Uh oh!

jiayisunx commented Apr 18, 2024

Uh oh!

pytorchmergebot commented Apr 18, 2024

Merge failed

Uh oh!

jiayisunx commented Apr 18, 2024

Uh oh!

pytorchmergebot commented Apr 18, 2024

Merge started

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

jiayisunx commented Mar 27, 2024 •

edited

Loading

pytorch-bot bot commented Mar 27, 2024 •

edited

Loading