inductor: align inductor behavior with eager mode for split_with_sizes #99702

XiaobingSuper · 2023-04-21T06:44:48Z

Stack from ghstack (oldest at bottom):

-> inductor: align inductor behavior with eager mode for split_with_sizes #99702
inductor: fix split+cat issue when cat order is not align the split output's order #99700

Fix #99686, for eager mode, if the given sizes is not meet requirements, it will report an error, but inductor can run, I think we need align inductor behavior with eager mode, the behavior will be like after this PR:

Traceback (most recent call last):
  File "/home/xiaobing/pytorch-offical/torch/_dynamo/utils.py", line 1267, in run_node
    return node.target(*args, **kwargs)
  File "/home/xiaobing/pytorch-offical/torch/functional.py", line 189, in split
    return tensor.split(split_size_or_sections, dim)
  File "/home/xiaobing/pytorch-offical/torch/_tensor.py", line 804, in split
    return torch._VF.split_with_sizes(self, split_size, dim)
  File "/home/xiaobing/pytorch-offical/torch/utils/_stats.py", line 20, in wrapper
    return fn(*args, **kwargs)
  File "/home/xiaobing/pytorch-offical/torch/_subclasses/fake_tensor.py", line 1095, in __torch_dispatch__
    return self.dispatch(func, types, args, kwargs)
  File "/home/xiaobing/pytorch-offical/torch/_subclasses/fake_tensor.py", line 1259, in dispatch
    return decomposition_table[func](*args, **kwargs)
  File "/home/xiaobing/pytorch-offical/torch/_decomp/decompositions.py", line 1102, in split_with_sizes
    raise ValueError(
ValueError: Split sizes don't add up to the tensor's size in the given dimension

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/xiaobing/pytorch-offical/torch/_dynamo/utils.py", line 1215, in get_fake_value
    return wrap_fake_exception(
  File "/home/xiaobing/pytorch-offical/torch/_dynamo/utils.py", line 835, in wrap_fake_exception
    return fn()
  File "/home/xiaobing/pytorch-offical/torch/_dynamo/utils.py", line 1216, in <lambda>
    lambda: run_node(tx.output, node, args, kwargs, nnmodule)
  File "/home/xiaobing/pytorch-offical/torch/_dynamo/utils.py", line 1279, in run_node
    raise RuntimeError(
RuntimeError: Failed running call_function <function split at 0x7f45b8402ee0>(*(FakeTensor(..., size=(1, 5)), [2, 1, 1]), **{'dim': 1}):
Split sizes don't add up to the tensor's size in the given dimension
(scroll up for backtrace)

The above exception was the direct cause of the following exception:

cc @soumith @voznesenskym @penguinwu @anijain2305 @EikanWang @jgong5 @Guobing-Chen @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @peterbell10 @desertfire

[ghstack-poisoned]

…t_with_sizes" Fix #99686, for eager mode, if the given sizes is not meet requirements, it will report an error, but inductor can run, I think we need align inductor behavior with eager mode, the behavior will be like after this PR: ``` Traceback (most recent call last): File "/home/xiaobing/pytorch-offical/torch/_dynamo/utils.py", line 1267, in run_node return node.target(*args, **kwargs) File "/home/xiaobing/pytorch-offical/torch/functional.py", line 189, in split return tensor.split(split_size_or_sections, dim) File "/home/xiaobing/pytorch-offical/torch/_tensor.py", line 804, in split return torch._VF.split_with_sizes(self, split_size, dim) File "/home/xiaobing/pytorch-offical/torch/utils/_stats.py", line 20, in wrapper return fn(*args, **kwargs) File "/home/xiaobing/pytorch-offical/torch/_subclasses/fake_tensor.py", line 1095, in __torch_dispatch__ return self.dispatch(func, types, args, kwargs) File "/home/xiaobing/pytorch-offical/torch/_subclasses/fake_tensor.py", line 1259, in dispatch return decomposition_table[func](*args, **kwargs) File "/home/xiaobing/pytorch-offical/torch/_decomp/decompositions.py", line 1102, in split_with_sizes raise ValueError( ValueError: Split sizes don't add up to the tensor's size in the given dimension The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/home/xiaobing/pytorch-offical/torch/_dynamo/utils.py", line 1215, in get_fake_value return wrap_fake_exception( File "/home/xiaobing/pytorch-offical/torch/_dynamo/utils.py", line 835, in wrap_fake_exception return fn() File "/home/xiaobing/pytorch-offical/torch/_dynamo/utils.py", line 1216, in <lambda> lambda: run_node(tx.output, node, args, kwargs, nnmodule) File "/home/xiaobing/pytorch-offical/torch/_dynamo/utils.py", line 1279, in run_node raise RuntimeError( RuntimeError: Failed running call_function <function split at 0x7f45b8402ee0>(*(FakeTensor(..., size=(1, 5)), [2, 1, 1]), **{'dim': 1}): Split sizes don't add up to the tensor's size in the given dimension (scroll up for backtrace) The above exception was the direct cause of the following exception: ``` cc soumith voznesenskym penguinwu anijain2305 EikanWang jgong5 Guobing-Chen zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx peterbell10 desertfire [ghstack-poisoned]

pytorch-bot · 2023-04-21T06:44:51Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/99702

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

ROCm jobs are failing with No space left on device

✅ No Failures

As of commit 8ab1336:
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

ghstack-source-id: 47d7d6517a1590fafa45b17f5375a3f51f970c8d Pull Request resolved: #99702

ghstack-source-id: 840658dc2b23a87a6162cd09a68c21e03a4babfa Pull Request resolved: #99702

…t_with_sizes" Fix #99686, for eager mode, if the given sizes is not meet requirements, it will report an error, but inductor can run, I think we need align inductor behavior with eager mode, the behavior will be like after this PR: ``` Traceback (most recent call last): File "/home/xiaobing/pytorch-offical/torch/_dynamo/utils.py", line 1267, in run_node return node.target(*args, **kwargs) File "/home/xiaobing/pytorch-offical/torch/functional.py", line 189, in split return tensor.split(split_size_or_sections, dim) File "/home/xiaobing/pytorch-offical/torch/_tensor.py", line 804, in split return torch._VF.split_with_sizes(self, split_size, dim) File "/home/xiaobing/pytorch-offical/torch/utils/_stats.py", line 20, in wrapper return fn(*args, **kwargs) File "/home/xiaobing/pytorch-offical/torch/_subclasses/fake_tensor.py", line 1095, in __torch_dispatch__ return self.dispatch(func, types, args, kwargs) File "/home/xiaobing/pytorch-offical/torch/_subclasses/fake_tensor.py", line 1259, in dispatch return decomposition_table[func](*args, **kwargs) File "/home/xiaobing/pytorch-offical/torch/_decomp/decompositions.py", line 1102, in split_with_sizes raise ValueError( ValueError: Split sizes don't add up to the tensor's size in the given dimension The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/home/xiaobing/pytorch-offical/torch/_dynamo/utils.py", line 1215, in get_fake_value return wrap_fake_exception( File "/home/xiaobing/pytorch-offical/torch/_dynamo/utils.py", line 835, in wrap_fake_exception return fn() File "/home/xiaobing/pytorch-offical/torch/_dynamo/utils.py", line 1216, in <lambda> lambda: run_node(tx.output, node, args, kwargs, nnmodule) File "/home/xiaobing/pytorch-offical/torch/_dynamo/utils.py", line 1279, in run_node raise RuntimeError( RuntimeError: Failed running call_function <function split at 0x7f45b8402ee0>(*(FakeTensor(..., size=(1, 5)), [2, 1, 1]), **{'dim': 1}): Split sizes don't add up to the tensor's size in the given dimension (scroll up for backtrace) The above exception was the direct cause of the following exception: ``` cc soumith voznesenskym penguinwu anijain2305 EikanWang jgong5 Guobing-Chen zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx peterbell10 desertfire [ghstack-poisoned]

jgong5 · 2023-04-21T07:29:11Z

torch/_decomp/decompositions.py

@@ -1087,10 +1087,21 @@ def prod(x: List[int]):
    return r


+def sum(x: List[int]):


why not use the builtin sum?

yes, changed.

jgong5 · 2023-04-21T07:30:31Z

test/inductor/test_torchinductor.py

+        def fn(a):
+            return torch.split(a, [2, 1, 1], dim=1)
+
+        with self.assertRaisesRegex(RuntimeError, ""):


Specify the regex pattern here.

Actually, shouldn't this be a ValueError?

I used ValueError, but it seems catch RuntimeError firstly even the error log is:

Traceback (most recent call last): File "/home/xiaobing/pytorch-offical/torch/_dynamo/utils.py", line 1267, in run_node return node.target(*args, **kwargs) File "/home/xiaobing/pytorch-offical/torch/functional.py", line 189, in split return tensor.split(split_size_or_sections, dim) File "/home/xiaobing/pytorch-offical/torch/_tensor.py", line 804, in split return torch._VF.split_with_sizes(self, split_size, dim) File "/home/xiaobing/pytorch-offical/torch/utils/_stats.py", line 20, in wrapper return fn(*args, **kwargs) File "/home/xiaobing/pytorch-offical/torch/_subclasses/fake_tensor.py", line 1095, in __torch_dispatch__ return self.dispatch(func, types, args, kwargs) File "/home/xiaobing/pytorch-offical/torch/_subclasses/fake_tensor.py", line 1259, in dispatch return decomposition_table[func](*args, **kwargs) File "/home/xiaobing/pytorch-offical/torch/_decomp/decompositions.py", line 1102, in split_with_sizes raise ValueError( ValueError: Split sizes don't add up to the tensor's size in the given dimension The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/home/xiaobing/pytorch-offical/torch/_dynamo/utils.py", line 1215, in get_fake_value return wrap_fake_exception( File "/home/xiaobing/pytorch-offical/torch/_dynamo/utils.py", line 835, in wrap_fake_exception return fn() File "/home/xiaobing/pytorch-offical/torch/_dynamo/utils.py", line 1216, in <lambda> lambda: run_node(tx.output, node, args, kwargs, nnmodule) File "/home/xiaobing/pytorch-offical/torch/_dynamo/utils.py", line 1279, in run_node raise RuntimeError( RuntimeError: Failed running call_function <function split at 0x7f45b8402ee0>(*(FakeTensor(..., size=(1, 5)), [2, 1, 1]), **{'dim': 1}): Split sizes don't add up to the tensor's size in the given dimension (scroll up for backtrace) The above exception was the direct cause of the following exception:

ghstack-source-id: 789217f504e980a424eb6567198081a610228fb7 Pull Request resolved: #99702

…t_with_sizes" Fix #99686, for eager mode, if the given sizes is not meet requirements, it will report an error, but inductor can run, I think we need align inductor behavior with eager mode, the behavior will be like after this PR: ``` Traceback (most recent call last): File "/home/xiaobing/pytorch-offical/torch/_dynamo/utils.py", line 1267, in run_node return node.target(*args, **kwargs) File "/home/xiaobing/pytorch-offical/torch/functional.py", line 189, in split return tensor.split(split_size_or_sections, dim) File "/home/xiaobing/pytorch-offical/torch/_tensor.py", line 804, in split return torch._VF.split_with_sizes(self, split_size, dim) File "/home/xiaobing/pytorch-offical/torch/utils/_stats.py", line 20, in wrapper return fn(*args, **kwargs) File "/home/xiaobing/pytorch-offical/torch/_subclasses/fake_tensor.py", line 1095, in __torch_dispatch__ return self.dispatch(func, types, args, kwargs) File "/home/xiaobing/pytorch-offical/torch/_subclasses/fake_tensor.py", line 1259, in dispatch return decomposition_table[func](*args, **kwargs) File "/home/xiaobing/pytorch-offical/torch/_decomp/decompositions.py", line 1102, in split_with_sizes raise ValueError( ValueError: Split sizes don't add up to the tensor's size in the given dimension The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/home/xiaobing/pytorch-offical/torch/_dynamo/utils.py", line 1215, in get_fake_value return wrap_fake_exception( File "/home/xiaobing/pytorch-offical/torch/_dynamo/utils.py", line 835, in wrap_fake_exception return fn() File "/home/xiaobing/pytorch-offical/torch/_dynamo/utils.py", line 1216, in <lambda> lambda: run_node(tx.output, node, args, kwargs, nnmodule) File "/home/xiaobing/pytorch-offical/torch/_dynamo/utils.py", line 1279, in run_node raise RuntimeError( RuntimeError: Failed running call_function <function split at 0x7f45b8402ee0>(*(FakeTensor(..., size=(1, 5)), [2, 1, 1]), **{'dim': 1}): Split sizes don't add up to the tensor's size in the given dimension (scroll up for backtrace) The above exception was the direct cause of the following exception: ``` cc soumith voznesenskym penguinwu anijain2305 EikanWang jgong5 Guobing-Chen zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx peterbell10 desertfire [ghstack-poisoned]

ghstack-source-id: e9597778b26b2a16a62cdf8912bcbd95fbfa899a Pull Request resolved: #99702

EikanWang · 2023-04-21T10:01:57Z

but inductor can run

Do you mean there is no correctness issue?

XiaobingSuper · 2023-04-23T01:19:10Z

but inductor can run

Do you mean there is no correctness issue?

For the definition of split_with_size, we should report an error to the user for the given case.

XiaobingSuper · 2023-04-25T01:11:35Z

@pytorchbot merge

pytorchmergebot · 2023-04-25T01:13:44Z

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

XiaobingSuper added 2 commits April 21, 2023 02:16

inductor: align inductor behavior with eager mode for split_with_sizes

c3e29d8

[ghstack-poisoned]

XiaobingSuper mentioned this pull request Apr 21, 2023

inductor: fix split+cat issue when cat order is not align the split output's order #99700

Closed

github-actions bot added the module: inductor label Apr 21, 2023

XiaobingSuper added a commit that referenced this pull request Apr 21, 2023

inductor: align inductor behavior with eager mode for split_with_sizes

20dd589

ghstack-source-id: 47d7d6517a1590fafa45b17f5375a3f51f970c8d Pull Request resolved: #99702

XiaobingSuper requested review from jansel, albanD and jgong5 April 21, 2023 06:45

XiaobingSuper added the release notes: inductor label Apr 21, 2023

pytorchbot added the open source label Apr 21, 2023

XiaobingSuper requested review from desertfire and removed request for albanD April 21, 2023 06:50

XiaobingSuper linked an issue Apr 21, 2023 that may be closed by this pull request

[torch.compile] WRONG VALUE for split+cat #99686

Closed

XiaobingSuper added a commit that referenced this pull request Apr 21, 2023

inductor: align inductor behavior with eager mode for split_with_sizes

892f0d5

ghstack-source-id: 840658dc2b23a87a6162cd09a68c21e03a4babfa Pull Request resolved: #99702

jgong5 requested changes Apr 21, 2023

View reviewed changes

XiaobingSuper added a commit that referenced this pull request Apr 21, 2023

inductor: align inductor behavior with eager mode for split_with_sizes

d4b8f32

ghstack-source-id: 789217f504e980a424eb6567198081a610228fb7 Pull Request resolved: #99702

XiaobingSuper added a commit that referenced this pull request Apr 21, 2023

inductor: align inductor behavior with eager mode for split_with_sizes

fdb6e2f

ghstack-source-id: e9597778b26b2a16a62cdf8912bcbd95fbfa899a Pull Request resolved: #99702

EikanWang self-requested a review April 21, 2023 10:02

XiaobingSuper requested a review from jgong5 April 24, 2023 05:36

lezcano approved these changes Apr 24, 2023

View reviewed changes

jgong5 approved these changes Apr 24, 2023

View reviewed changes

XiaobingSuper added the ciflow/trunk Trigger trunk jobs on your pull request label Apr 24, 2023

jansel approved these changes Apr 24, 2023

View reviewed changes

pytorchmergebot added the merging label Apr 25, 2023

pytorchmergebot added the Merged label Apr 25, 2023

pytorchmergebot closed this in 41069f2 Apr 25, 2023

facebook-github-bot deleted the gh/XiaobingSuper/100/head branch June 8, 2023 14:58

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

inductor: align inductor behavior with eager mode for split_with_sizes #99702

inductor: align inductor behavior with eager mode for split_with_sizes #99702

XiaobingSuper commented Apr 21, 2023 •

edited

pytorch-bot bot commented Apr 21, 2023 •

edited

jgong5 Apr 21, 2023

XiaobingSuper Apr 21, 2023

jgong5 Apr 21, 2023

lezcano Apr 24, 2023

XiaobingSuper Apr 24, 2023

lezcano Apr 24, 2023

EikanWang commented Apr 21, 2023

XiaobingSuper commented Apr 23, 2023

XiaobingSuper commented Apr 25, 2023

pytorchmergebot commented Apr 25, 2023

		@@ -1087,10 +1087,21 @@ def prod(x: List[int]):
		return r


		def sum(x: List[int]):

inductor: align inductor behavior with eager mode for split_with_sizes #99702

inductor: align inductor behavior with eager mode for split_with_sizes #99702

Conversation

XiaobingSuper commented Apr 21, 2023 • edited

pytorch-bot bot commented Apr 21, 2023 • edited

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/99702

❗ 1 Active SEVs

✅ No Failures

jgong5 Apr 21, 2023

Choose a reason for hiding this comment

XiaobingSuper Apr 21, 2023

Choose a reason for hiding this comment

jgong5 Apr 21, 2023

Choose a reason for hiding this comment

lezcano Apr 24, 2023

Choose a reason for hiding this comment

XiaobingSuper Apr 24, 2023

Choose a reason for hiding this comment

lezcano Apr 24, 2023

Choose a reason for hiding this comment

EikanWang commented Apr 21, 2023

XiaobingSuper commented Apr 23, 2023

XiaobingSuper commented Apr 25, 2023

pytorchmergebot commented Apr 25, 2023

Merge started

XiaobingSuper commented Apr 21, 2023 •

edited

pytorch-bot bot commented Apr 21, 2023 •

edited