[quant][graphmode] Invoke more passes in insertObservers
#30473
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Summary: Invoked `ConstantPooling` and `FuseLinear` pass before `insertObservers`. `ConstantPooling` is for cleanning up traced graph, e.g. when we have to constant node that has the same value, this pass will merge them, this allows us to have less quantization patterns `FuseLinear` is to merge the exploded linear function into `aten::linear` so that we can quantize this function properly. We need to fuse it because right now the way we recognize weight and bias is by matching the argument position in certain function calls, e.g. 1st argument of aten::conv2d is weight. Therefore we have to preserve the bounary of the linear function to recognize the weight of linear. Since in the exploded linear code, input of addmm is transposed weight rather than the original weight of linear. Test Plan: This is needed for quantizing traced model tests to pass Reviewers: mvz Subscribers: Tasks: Tags: [ghstack-poisoned]
facebook-github-bot
added
the
oncall: jit
This was referenced Nov 26, 2019
| auto graph = method.graph(); | ||
| ConstantPropagation(graph); | ||
| // To cleanup traced graph | ||
| ConstantPooling(graph); |
There was a problem hiding this comment.
Just curious, is there any order we can follow for the optimization passes?
There was a problem hiding this comment.
not sure, looks like in graph_executor constant pooling is done before constant propagation, I'll change the order
There was a problem hiding this comment.
If we want to cleanup traced graphs, shouldn't we do it in tracing?
There was a problem hiding this comment.
Not sure why we don't do this in traced graph, cc @jamesr66a
There was a problem hiding this comment.
Was speaking with @suo yesterday. It's a pretty common request to add more "clean-up" passes to Module emission (i.e. tracing or scripting) as opposed to just in the GraphExecutor. I think this PR is good, but it would also be valid to just add these things to tracing
ZolotukhinM
reviewed
Dec 2, 2019
| auto graph = method.graph(); | ||
| ConstantPropagation(graph); | ||
| // To cleanup traced graph | ||
| ConstantPooling(graph); |
There was a problem hiding this comment.
If we want to cleanup traced graphs, shouldn't we do it in tracing?
Summary: Invoked `ConstantPooling` and `FuseLinear` pass before `insertObservers`. `ConstantPooling` is for cleanning up traced graph, e.g. when we have to constant node that has the same value, this pass will merge them, this allows us to have less quantization patterns `FuseLinear` is to merge the exploded linear function into `aten::linear` so that we can quantize this function properly. We need to fuse it because right now the way we recognize weight and bias is by matching the argument position in certain function calls, e.g. 1st argument of aten::conv2d is weight. Therefore we have to preserve the bounary of the linear function to recognize the weight of linear. Since in the exploded linear code, input of addmm is transposed weight rather than the original weight of linear. Test Plan: This is needed for quantizing traced model tests to pass Reviewers: mvz Subscribers: Tasks: Tags: [ghstack-poisoned]
Summary: Invoked `ConstantPooling` and `FuseLinear` pass before `insertObservers`. `ConstantPooling` is for cleanning up traced graph, e.g. when we have to constant node that has the same value, this pass will merge them, this allows us to have less quantization patterns `FuseLinear` is to merge the exploded linear function into `aten::linear` so that we can quantize this function properly. We need to fuse it because right now the way we recognize weight and bias is by matching the argument position in certain function calls, e.g. 1st argument of aten::conv2d is weight. Therefore we have to preserve the bounary of the linear function to recognize the weight of linear. Since in the exploded linear code, input of addmm is transposed weight rather than the original weight of linear. Test Plan: This is needed for quantizing traced model tests to pass Reviewers: mvz Subscribers: Tasks: Tags: [ghstack-poisoned]
Summary: Invoked `ConstantPooling` and `FuseLinear` pass before `insertObservers`. `ConstantPooling` is for cleanning up traced graph, e.g. when we have to constant node that has the same value, this pass will merge them, this allows us to have less quantization patterns `FuseLinear` is to merge the exploded linear function into `aten::linear` so that we can quantize this function properly. We need to fuse it because right now the way we recognize weight and bias is by matching the argument position in certain function calls, e.g. 1st argument of aten::conv2d is weight. Therefore we have to preserve the bounary of the linear function to recognize the weight of linear. Since in the exploded linear code, input of addmm is transposed weight rather than the original weight of linear. Test Plan: This is needed for quantizing traced model tests to pass Reviewers: mvz Subscribers: Tasks: Tags: Differential Revision: [D18795722](https://our.internmc.facebook.com/intern/diff/D18795722) [ghstack-poisoned]
jerryzh168
added a commit
that referenced
this pull request
Dec 4, 2019
Pull Request resolved: #30473 Invoked `ConstantPooling` and `FuseLinear` pass before `insertObservers`. `ConstantPooling` is for cleanning up traced graph, e.g. when we have to constant node that has the same value, this pass will merge them, this allows us to have less quantization patterns `FuseLinear` is to merge the exploded linear function into `aten::linear` so that we can quantize this function properly. We need to fuse it because right now the way we recognize weight and bias is by matching the argument position in certain function calls, e.g. 1st argument of aten::conv2d is weight. Therefore we have to preserve the bounary of the linear function to recognize the weight of linear. Since in the exploded linear code, input of addmm is transposed weight rather than the original weight of linear. Differential Revision: [D18795722](https://our.internmc.facebook.com/intern/diff/D18795722/) ghstack-source-id: 94887831
jamesr66a
approved these changes
Dec 4, 2019
| auto graph = method.graph(); | ||
| ConstantPropagation(graph); | ||
| // To cleanup traced graph | ||
| ConstantPooling(graph); |
There was a problem hiding this comment.
Was speaking with @suo yesterday. It's a pretty common request to add more "clean-up" passes to Module emission (i.e. tracing or scripting) as opposed to just in the GraphExecutor. I think this PR is good, but it would also be valid to just add these things to tracing
| // must do constant propagation first before replacement | ||
| replaceConvolutionWithConv2d(graph); | ||
| // fuse decomposed linear into aten::linear | ||
| FuseLinear(graph); |
There was a problem hiding this comment.
Though for this pass I remember there was some back-and-forth earlier about how it's not profitable in some cases, and might mess up some downstream systems (e.g. CUDA fuser). We could add it generally and see if anything breaks
There was a problem hiding this comment.
yeah we can't do this by default since it doesn't work for autodiff, but @wanchaol is working on this.
Summary: Invoked `ConstantPooling` and `FuseLinear` pass before `insertObservers`. `ConstantPooling` is for cleanning up traced graph, e.g. when we have to constant node that has the same value, this pass will merge them, this allows us to have less quantization patterns `FuseLinear` is to merge the exploded linear function into `aten::linear` so that we can quantize this function properly. We need to fuse it because right now the way we recognize weight and bias is by matching the argument position in certain function calls, e.g. 1st argument of aten::conv2d is weight. Therefore we have to preserve the bounary of the linear function to recognize the weight of linear. Since in the exploded linear code, input of addmm is transposed weight rather than the original weight of linear. Test Plan: This is needed for quantizing traced model tests to pass Reviewers: mvz Subscribers: Tasks: Tags: Differential Revision: [D18795722](https://our.internmc.facebook.com/intern/diff/D18795722) [ghstack-poisoned]
wuhuikx
pushed a commit
to wuhuikx/pytorch
that referenced
this pull request
Jan 30, 2020
Summary: Pull Request resolved: pytorch#30473 Invoked `ConstantPooling` and `FuseLinear` pass before `insertObservers`. `ConstantPooling` is for cleanning up traced graph, e.g. when we have to constant node that has the same value, this pass will merge them, this allows us to have less quantization patterns `FuseLinear` is to merge the exploded linear function into `aten::linear` so that we can quantize this function properly. We need to fuse it because right now the way we recognize weight and bias is by matching the argument position in certain function calls, e.g. 1st argument of aten::conv2d is weight. Therefore we have to preserve the bounary of the linear function to recognize the weight of linear. Since in the exploded linear code, input of addmm is transposed weight rather than the original weight of linear. ghstack-source-id: 94887831 Test Plan: This is needed for quantizing traced model tests to pass Imported from OSS Differential Revision: D18795722 fbshipit-source-id: 192d9d1e56307e2e1d90e30dce0502e31cb4f829
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Stack from ghstack:
test_nested#30475 Refactor test_quantization.py and enabletest_nestedinsertObservers#30473 [quant][graphmode] Invoke more passes ininsertObserversSummary:
Invoked
ConstantPoolingandFuseLinearpass beforeinsertObservers.ConstantPoolingis for cleanning up traced graph, e.g. when wehave to constant node that has the same value, this pass will merge them,
this allows us to have less quantization patterns
FuseLinearis to merge the exploded linear function intoaten::linearsothat we can quantize this function properly. We need to fuse it because right now
the way we recognize weight and bias is by matching the argument position in certain function
calls, e.g. 1st argument of aten::conv2d is weight. Therefore we have to preserve
the bounary of the linear function to recognize the weight of linear. Since in the exploded
linear code, input of addmm is transposed weight rather than the original weight of linear.
Test Plan:
This is needed for quantizing traced model tests to pass
Reviewers:
mvz
Subscribers:
Tasks:
Tags:
Differential Revision: D18795722