[ONNX] Support converting fx graph with symbolic shape to ONNX #96350
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
[ghstack-poisoned]
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/96350
Note: Links to docs will display an error until the docs builds have been completed. ✅ No FailuresAs of commit 030c219: This comment was automatically generated by Dr. CI and updates every 15 minutes. |
titaiwangms
added a commit
that referenced
this pull request
Mar 8, 2023
ghstack-source-id: 8275398f84468b91e4b32475bb467199ad65f4cc Pull Request resolved: #96350
[ghstack-poisoned]
titaiwangms
added a commit
that referenced
this pull request
Mar 8, 2023
ghstack-source-id: c97707fc2b99535defffcd9a81e63bf7fdbf8571 Pull Request resolved: #96350
[ghstack-poisoned]
titaiwangms
added a commit
that referenced
this pull request
Mar 9, 2023
ghstack-source-id: aa1271240b32e06e0afd82a9c04534f02cb436b7 Pull Request resolved: #96350
[ghstack-poisoned]
titaiwangms
added a commit
that referenced
this pull request
Mar 9, 2023
ghstack-source-id: ad4da6f942213a7a61bc32e3cbef6ed2f4ca8243 Pull Request resolved: #96350
[ghstack-poisoned]
titaiwangms
added a commit
that referenced
this pull request
Mar 9, 2023
ghstack-source-id: c14e4740e7e4ea00aaae679740a3172e5b27b318 Pull Request resolved: #96350
[ghstack-poisoned]
titaiwangms
added a commit
that referenced
this pull request
Mar 10, 2023
ghstack-source-id: 0bacec8d5cf34f4a715ebf19dd5746716cda477a Pull Request resolved: #96350
[ghstack-poisoned]
titaiwangms
added a commit
that referenced
this pull request
Mar 13, 2023
ghstack-source-id: 8f60f39e62337a60ef57a4460078fca6c7a8c26d Pull Request resolved: #96350
[ghstack-poisoned]
titaiwangms
added a commit
that referenced
this pull request
Mar 13, 2023
ghstack-source-id: ee6454b650d3bf885e8b3d8328c6f99455d5ceb3 Pull Request resolved: #96350
[ghstack-poisoned]
titaiwangms
added a commit
that referenced
this pull request
Mar 13, 2023
ghstack-source-id: ae6809fd2fb4108539594ed4d401f1026fcdd6bb Pull Request resolved: #96350
titaiwangms
added
module: onnx
wschin
reviewed
Mar 22, 2023
wschin
reviewed
Mar 22, 2023
wschin
reviewed
Mar 22, 2023
wschin
reviewed
Mar 22, 2023
BowenBao
reviewed
Mar 22, 2023
…ONNX" ~~Need microsoft/onnxscript#484 Support dynamic export on fx-ONNX exporter. Essentially, we set inputs size and nodes all dynamic in torchscript, and leverage on `aten::sym_size` to catch dynamic size between each Op. 1. Add `dynamic_axes` switch between symbolic tracing (dynamic sizes) and fake mode (static). Set it to default True, as most of our tests are happy with sumbolic tracing. Except GPT2 stays with fake mode with error: microsoft/onnxscript#523 2. Add test_fx_dynamic_onnruntime.py to test on some addhoc we have from old exporter. This can be removed once tests are integrated with #96479 3. Since `aten::sym_size` are operated with built-in function, a built-in function mapping is added to support SymFloat/SymInt. (FIXME: #97201). sym_size output value is also fx.Node, and can be found in `fx_name_to_onnxscipt_value`, so it's operation stays the same as other ONNX ops in ONNX graph. 4. Fully deprecated FakeTensorProp as make_fx() should provide all node meta info. 5. Put complicated fx.Node related ArgumentType into _type_utils.py [ghstack-poisoned]
titaiwangms
added a commit
that referenced
this pull request
Mar 22, 2023
ghstack-source-id: baca7d89ada271af08376ab607325d89b868471c Pull Request resolved: #96350 --amend
|
@wschin PTAL |
wschin
reviewed
Mar 24, 2023
wschin
approved these changes
Mar 24, 2023
…ONNX" ~~Need microsoft/onnxscript#484 Support dynamic export on fx-ONNX exporter. Essentially, we set inputs size and nodes all dynamic in torchscript, and leverage on `aten::sym_size` to catch dynamic size between each Op. 1. Add `dynamic_axes` switch between symbolic tracing (dynamic sizes) and fake mode (static). Set it to default True, as most of our tests are happy with sumbolic tracing. Except GPT2 stays with fake mode with error: microsoft/onnxscript#523 2. Add test_fx_dynamic_onnruntime.py to test on some addhoc we have from old exporter. This can be removed once tests are integrated with #96479 3. Since `aten::sym_size` are operated with built-in function, a built-in function mapping is added to support SymFloat/SymInt. (FIXME: #97201). sym_size output value is also fx.Node, and can be found in `fx_name_to_onnxscipt_value`, so it's operation stays the same as other ONNX ops in ONNX graph. 4. Fully deprecated FakeTensorProp as make_fx() should provide all node meta info. 5. Put complicated fx.Node related ArgumentType into _type_utils.py [ghstack-poisoned]
…ONNX" ~~Need microsoft/onnxscript#484 Support dynamic export on fx-ONNX exporter. Essentially, we set inputs size and nodes all dynamic in torchscript, and leverage on `aten::sym_size` to catch dynamic size between each Op. 1. Add `dynamic_axes` switch between symbolic tracing (dynamic sizes) and fake mode (static). Set it to default True, as most of our tests are happy with sumbolic tracing. Except GPT2 stays with fake mode with error: microsoft/onnxscript#523 2. Add test_fx_dynamic_onnruntime.py to test on some addhoc we have from old exporter. This can be removed once tests are integrated with #96479 3. Since `aten::sym_size` are operated with built-in function, a built-in function mapping is added to support SymFloat/SymInt. (FIXME: #97201). sym_size output value is also fx.Node, and can be found in `fx_name_to_onnxscipt_value`, so it's operation stays the same as other ONNX ops in ONNX graph. 4. Fully deprecated FakeTensorProp as make_fx() should provide all node meta info. 5. Put complicated fx.Node related ArgumentType into _type_utils.py [ghstack-poisoned]
…ONNX" ~~Need microsoft/onnxscript#484 Support dynamic export on fx-ONNX exporter. Essentially, we set inputs size and nodes all dynamic in torchscript, and leverage on `aten::sym_size` to catch dynamic size between each Op. 1. Add `dynamic_axes` switch between symbolic tracing (dynamic sizes) and fake mode (static). Set it to default True, as most of our tests are happy with sumbolic tracing. Except GPT2 stays with fake mode with error: microsoft/onnxscript#523 2. Add test_fx_dynamic_onnruntime.py to test on some addhoc we have from old exporter. This can be removed once tests are integrated with #96479 3. Since `aten::sym_size` are operated with built-in function, a built-in function mapping is added to support SymFloat/SymInt. (FIXME: #97201). sym_size output value is also fx.Node, and can be found in `fx_name_to_onnxscipt_value`, so it's operation stays the same as other ONNX ops in ONNX graph. 4. Fully deprecated FakeTensorProp as make_fx() should provide all node meta info. 5. Put complicated fx.Node related ArgumentType into _type_utils.py [ghstack-poisoned]
|
@pytorchbot merge |
|
❌ 🤖 pytorchbot command failed: Try |
|
@pytorchbot merge |
Merge startedYour change will be merged once all checks pass (ETA 0-4 Hours). Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
Merge failedReason: 1 mandatory check(s) failed. The first few are: Dig deeper by viewing the failures on hud |
|
@pytorchbot merge |
Merge startedYour change will be merged once all checks pass (ETA 0-4 Hours). Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
|
Great work 🙌 |
airen3339
pushed a commit
to airen3339/pytorch
that referenced
this pull request
Apr 19, 2023
ghstack-source-id: fc50126f92dd2b2a5468b42bcdd299b54e313319 Pull Request resolved: pytorch/pytorch#96350 --amend
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Stack from ghstack (oldest at bottom):
Need microsoft/onnxscript#484Support dynamic export on fx-ONNX exporter. Essentially, we set inputs size and nodes all dynamic in torchscript, and leverage on
aten::sym_sizeto catch dynamic size between each Op.dynamic_axesswitch between symbolic tracing (dynamic sizes) and fake mode (static). Set it to default True, as most of our tests are happy with sumbolic tracing. Except GPT2 stays with fake mode with error: ORT Flaky Segmentation Fault during model load microsoft/onnxscript#523aten::sym_sizeare operated with built-in function, a built-in function mapping is added to support SymFloat/SymInt. (FIXME: aten::sym_size is not using torch._ops.OpOverload in FX graph #97201). sym_size output value is also fx.Node, and can be found infx_name_to_onnxscipt_value, so it's operation stays the same as other ONNX ops in ONNX graph.