[quant][pt2][be] Rewrite QAT annotations using subgraph matcher #113709
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Summary: This is the recommended way to write quantizers according to https://pytorch.org/tutorials/prototype/pt2e_quantizer.html#a-note-on-ir-for-pt2e-quantization-flow. It is agnostic to changes in the aten IR and can be easily extended to support conv1d-bn and conv3d-bn fusion patterns in the future. This is the first step towards rewriting XNNPACKQuantizer using this subgraph matcher. Test Plan: python test/test_quantization.py TestQuantizePT2EQAT_ConvBn2d Reviewers: jerryzh168, kimishpatel Subscribers: jerryzh168, kimishpatel, supriyar [ghstack-poisoned]
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/113709
Note: Links to docs will display an error until the docs builds have been completed. ✅ No FailuresAs of commit 89df0e4 with merge base 05d9492 ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
andrewor14
added a commit
that referenced
this pull request
Nov 14, 2023
Summary: This is the recommended way to write quantizers according to https://pytorch.org/tutorials/prototype/pt2e_quantizer.html#a-note-on-ir-for-pt2e-quantization-flow. It is agnostic to changes in the aten IR and can be easily extended to support conv1d-bn and conv3d-bn fusion patterns in the future. This is the first step towards rewriting XNNPACKQuantizer using this subgraph matcher. Test Plan: python test/test_quantization.py TestQuantizePT2EQAT_ConvBn2d Reviewers: jerryzh168, kimishpatel Subscribers: jerryzh168, kimishpatel, supriyar ghstack-source-id: de45e114d20c21064b5d0869aed8336c70a423e8 Pull Request resolved: #113709
…tcher" Summary: This is the recommended way to write quantizers according to https://pytorch.org/tutorials/prototype/pt2e_quantizer.html#a-note-on-ir-for-pt2e-quantization-flow. It is agnostic to changes in the aten IR and can be easily extended to support conv1d-bn and conv3d-bn fusion patterns in the future. This is the first step towards rewriting XNNPACKQuantizer using this subgraph matcher. Test Plan: python test/test_quantization.py TestQuantizePT2EQAT_ConvBn2d Reviewers: jerryzh168, kimishpatel Subscribers: jerryzh168, kimishpatel, supriyar [ghstack-poisoned]
andrewor14
added a commit
that referenced
this pull request
Nov 14, 2023
Summary: This is the recommended way to write quantizers according to https://pytorch.org/tutorials/prototype/pt2e_quantizer.html#a-note-on-ir-for-pt2e-quantization-flow. It is agnostic to changes in the aten IR and can be easily extended to support conv1d-bn and conv3d-bn fusion patterns in the future. This is the first step towards rewriting XNNPACKQuantizer using this subgraph matcher. Test Plan: python test/test_quantization.py TestQuantizePT2EQAT_ConvBn2d Reviewers: jerryzh168, kimishpatel Subscribers: jerryzh168, kimishpatel, supriyar ghstack-source-id: d6797c849f4a2ceda94e3d95b4d908d9c746be99 Pull Request resolved: #113709
jerryzh168
reviewed
Nov 14, 2023
jerryzh168
reviewed
Nov 14, 2023
…tcher" Summary: This is the recommended way to write quantizers according to https://pytorch.org/tutorials/prototype/pt2e_quantizer.html#a-note-on-ir-for-pt2e-quantization-flow. It is agnostic to changes in the aten IR and can be easily extended to support conv1d-bn and conv3d-bn fusion patterns in the future. This is the first step towards rewriting XNNPACKQuantizer using this subgraph matcher. Test Plan: python test/test_quantization.py TestQuantizePT2EQAT_ConvBn2d Reviewers: jerryzh168, kimishpatel Subscribers: jerryzh168, kimishpatel, supriyar [ghstack-poisoned]
…tcher" Summary: This is the recommended way to write quantizers according to https://pytorch.org/tutorials/prototype/pt2e_quantizer.html#a-note-on-ir-for-pt2e-quantization-flow. It is agnostic to changes in the aten IR and can be easily extended to support conv1d-bn and conv3d-bn fusion patterns in the future. This is the first step towards rewriting XNNPACKQuantizer using this subgraph matcher. Test Plan: python test/test_quantization.py TestQuantizePT2EQAT_ConvBn2d Reviewers: jerryzh168, kimishpatel Subscribers: jerryzh168, kimishpatel, supriyar [ghstack-poisoned]
jerryzh168
reviewed
Nov 15, 2023
| weight_user = list(weight_node.users.keys())[0] | ||
| if weight_user is not input_user: | ||
| raise ValueError("Expected weight user to be the same as input user") | ||
| conv_node = input_user |
There was a problem hiding this comment.
This still assumes conv_node is not decomposed. (when it's decomposed, input_user, weight_user and bias_user could be different nodes, e.g. linear is a good example)
I think you'll need to annotate input_user, weight_user and bias_user separately to be 100% robust to decompositions of the conv op
There was a problem hiding this comment.
As discussed offline, we will just use the conv_node returned from the subgraph matcher here and just validate the conv args for now. This is because we can't always rely on the first user of the input node, for example in resnet18 where there are skip connections
…tcher" Summary: This is the recommended way to write quantizers according to https://pytorch.org/tutorials/prototype/pt2e_quantizer.html#a-note-on-ir-for-pt2e-quantization-flow. It is agnostic to changes in the aten IR and can be easily extended to support conv1d-bn and conv3d-bn fusion patterns in the future. This is the first step towards rewriting XNNPACKQuantizer using this subgraph matcher. Test Plan: python test/test_quantization.py TestQuantizePT2EQAT_ConvBn2d Reviewers: jerryzh168, kimishpatel Subscribers: jerryzh168, kimishpatel, supriyar [ghstack-poisoned]
…tcher" Summary: This is the recommended way to write quantizers according to https://pytorch.org/tutorials/prototype/pt2e_quantizer.html#a-note-on-ir-for-pt2e-quantization-flow. It is agnostic to changes in the aten IR and can be easily extended to support conv1d-bn and conv3d-bn fusion patterns in the future. This is the first step towards rewriting XNNPACKQuantizer using this subgraph matcher. Test Plan: python test/test_quantization.py TestQuantizePT2EQAT_ConvBn2d Reviewers: jerryzh168, kimishpatel Subscribers: jerryzh168, kimishpatel, supriyar [ghstack-poisoned]
|
@andrewor14 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
…tcher" Summary: This is the recommended way to write quantizers according to https://pytorch.org/tutorials/prototype/pt2e_quantizer.html#a-note-on-ir-for-pt2e-quantization-flow. It is agnostic to changes in the aten IR and can be easily extended to support conv1d-bn and conv3d-bn fusion patterns in the future. This is the first step towards rewriting XNNPACKQuantizer using this subgraph matcher. Test Plan: python test/test_quantization.py TestQuantizePT2EQAT_ConvBn2d Reviewers: jerryzh168, kimishpatel Subscribers: jerryzh168, kimishpatel, supriyar Differential Revision: [D51366525](https://our.internmc.facebook.com/intern/diff/D51366525) [ghstack-poisoned]
|
@andrewor14 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
jerryzh168
approved these changes
Nov 16, 2023
|
@pytorchbot merge |
Merge startedYour change will be merged once all checks pass (ETA 0-4 Hours). Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Stack from ghstack (oldest at bottom):
Summary: This is the recommended way to write quantizers according
to https://pytorch.org/tutorials/prototype/pt2e_quantizer.html#a-note-on-ir-for-pt2e-quantization-flow.
It is agnostic to changes in the aten IR and can be easily extended
to support conv1d-bn and conv3d-bn fusion patterns in the future.
This is the first step towards rewriting XNNPACKQuantizer using
this subgraph matcher.
Test Plan:
python test/test_quantization.py TestQuantizePT2EQAT_ConvBn2d
Reviewers: jerryzh168, kimishpatel
Subscribers: jerryzh168, kimishpatel, supriyar
Differential Revision: D51366525