[onnx][quantization] Add JIT pass to insert permutes for conv ops #30679
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Summary: Caffe2 expects quantized ops to be in NHWC format while pytorch inputs are in NCHW. Add a jit pass to insert permutes to convert from nchw2nhwc before each conv op and add nhwc2nchw permute after the conv op. Using graph rewriter to find consecutive redundant permutes and remove them from the graph Test Plan: python test/onnx/test_pytorch_onnx_caffe2_quantized.py TestQuantizedOps Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned]
facebook-github-bot
added
the
oncall: jit
…onv ops" Summary: Caffe2 expects quantized ops to be in NHWC format while pytorch inputs are in NCHW. Add a jit pass to insert permutes to convert from nchw2nhwc before each conv op and add nhwc2nchw permute after the conv op. Using graph rewriter to find consecutive redundant permutes and remove them from the graph Test Plan: python test/onnx/test_pytorch_onnx_caffe2_quantized.py TestQuantizedOps Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned]
supriyar
added a commit
that referenced
this pull request
Dec 3, 2019
Summary: Caffe2 expects quantized ops to be in NHWC format while pytorch inputs are in NCHW. Add a jit pass to insert permutes to convert from nchw2nhwc before each conv op and add nhwc2nchw permute after the conv op. Using graph rewriter to find consecutive redundant permutes and remove them from the graph Test Plan: python test/onnx/test_pytorch_onnx_caffe2_quantized.py TestQuantizedOps Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 889aef9 Pull Request resolved: #30679
…onv ops" Summary: Caffe2 expects quantized ops to be in NHWC format while pytorch inputs are in NCHW. Add a jit pass to insert permutes to convert from nchw2nhwc before each conv op and add nhwc2nchw permute after the conv op. Using graph rewriter to find consecutive redundant permutes and remove them from the graph Test Plan: python test/onnx/test_pytorch_onnx_caffe2_quantized.py TestQuantizedOps Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D18790518](https://our.internmc.facebook.com/intern/diff/D18790518) [ghstack-poisoned]
supriyar
added a commit
that referenced
this pull request
Dec 3, 2019
Summary: Caffe2 expects quantized ops to be in NHWC format while pytorch inputs are in NCHW. Add a jit pass to insert permutes to convert from nchw2nhwc before each conv op and add nhwc2nchw permute after the conv op. Using graph rewriter to find consecutive redundant permutes and remove them from the graph Test Plan: python test/onnx/test_pytorch_onnx_caffe2_quantized.py TestQuantizedOps Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 190567f Pull Request resolved: #30679
…onv ops" Summary: Caffe2 expects quantized ops to be in NHWC format while pytorch inputs are in NCHW. Add a jit pass to insert permutes to convert from nchw2nhwc before each conv op and add nhwc2nchw permute after the conv op. Using graph rewriter to find consecutive redundant permutes and remove them from the graph Test Plan: python test/onnx/test_pytorch_onnx_caffe2_quantized.py TestQuantizedOps Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D18790518](https://our.internmc.facebook.com/intern/diff/D18790518) [ghstack-poisoned]
supriyar
added a commit
that referenced
this pull request
Dec 4, 2019
Summary: Caffe2 expects quantized ops to be in NHWC format while pytorch inputs are in NCHW. Add a jit pass to insert permutes to convert from nchw2nhwc before each conv op and add nhwc2nchw permute after the conv op. Using graph rewriter to find consecutive redundant permutes and remove them from the graph Test Plan: python test/onnx/test_pytorch_onnx_caffe2_quantized.py TestQuantizedOps Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 68d1d19 Pull Request resolved: #30679
houseroad
reviewed
Dec 5, 2019
| torch._C._jit_pass_onnx_unpack_quantized_weights(graph, params_dict) | ||
|
|
||
| # Insert permutes before and after each conv op to ensure correct order. | ||
| torch._C._jit_pass_onnx_quantization_insert_permutes(graph, params_dict) |
There was a problem hiding this comment.
Is it possible we can use _jit_pass_custom_pattern_based_rewrite_graph to insert permutes as well?
There was a problem hiding this comment.
I did try that but got an assert of the form
terminate called after throwing an instance of 'c10::Error'
what(): n->owningGraph() == this && n->inBlockList() INTERNAL ASSERT FAILED at ../torch/csrc/jit/ir.h:1191
I suspect adding new ops to the graph using the rewriter isn't updating all the required dependencies correctly.
| op_node->insertInput(0, permute_node->output()); | ||
|
|
||
| Node* permute_node_after = graph->create( | ||
| Symbol::fromQualString("quantized::nhwc2nchw"), {input_node->output()}); |
There was a problem hiding this comment.
why use input_node->output()? shall we use op_node->outputs()[0] here?
…onv ops" Summary: Caffe2 expects quantized ops to be in NHWC format while pytorch inputs are in NCHW. Add a jit pass to insert permutes to convert from nchw2nhwc before each conv op and add nhwc2nchw permute after the conv op. Using graph rewriter to find consecutive redundant permutes and remove them from the graph Test Plan: python test/onnx/test_pytorch_onnx_caffe2_quantized.py TestQuantizedOps Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D18790518](https://our.internmc.facebook.com/intern/diff/D18790518) [ghstack-poisoned]
supriyar
added a commit
that referenced
this pull request
Dec 5, 2019
Summary: Caffe2 expects quantized ops to be in NHWC format while pytorch inputs are in NCHW. Add a jit pass to insert permutes to convert from nchw2nhwc before each conv op and add nhwc2nchw permute after the conv op. Using graph rewriter to find consecutive redundant permutes and remove them from the graph Test Plan: python test/onnx/test_pytorch_onnx_caffe2_quantized.py TestQuantizedOps Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 22f7a0b Pull Request resolved: #30679
|
This pull request has been merged in a51c5f5. |
wuhuikx
pushed a commit
to wuhuikx/pytorch
that referenced
this pull request
Jan 30, 2020
Summary: Pull Request resolved: pytorch#30679 Caffe2 expects quantized ops to be in NHWC format while pytorch inputs are in NCHW. Add a jit pass to insert permutes to convert from nchw2nhwc before each conv op and add nhwc2nchw permute after the conv op. Using graph rewriter to find consecutive redundant permutes and remove them from the graph Test Plan: python test/onnx/test_pytorch_onnx_caffe2_quantized.py TestQuantizedOps Imported from OSS Differential Revision: D18790518 fbshipit-source-id: 4dd39cf0b31b21f5586c0edfdce2260d4e245112
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Stack from ghstack:
Summary:
Caffe2 expects quantized ops to be in NHWC format while pytorch inputs are in NCHW.
Add a jit pass to insert permutes to convert from nchw2nhwc before each conv op and add nhwc2nchw permute after the conv op.
Using graph rewriter to find consecutive redundant permutes and remove them from the graph
Test Plan:
python test/onnx/test_pytorch_onnx_caffe2_quantized.py TestQuantizedOps
Reviewers:
Subscribers:
Tasks:
Tags:
Differential Revision: D18790518