[quant][graphmode][fx] Support sigmoid/hardsigmoid/tanh in qat #46738
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Summary: Test Plan: Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned]
… qat" Summary: Test Plan: Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D24486972](https://our.internmc.facebook.com/intern/diff/D24486972) [ghstack-poisoned]
jerryzh168
added a commit
that referenced
this pull request
Oct 22, 2020
Summary: Test Plan: Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: b44738c8fde57db49edb77e223df0a1fc7ae8840 Pull Request resolved: #46738
💊 CI failures summary and remediationsAs of commit 4a1d3b6 (more details on the Dr. CI page):
Extra GitHub checks: 1 failed
This comment was automatically generated by Dr. CI (expand for details).Follow this link to opt-out of these comments for your Pull Requests.Please report bugs/suggestions on the GitHub issue tracker or post in the (internal) Dr. CI Users group. This comment has been revised 33 times. |
… qat" Summary: Test Plan: Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D24486972](https://our.internmc.facebook.com/intern/diff/D24486972) [ghstack-poisoned]
jerryzh168
added a commit
that referenced
this pull request
Oct 23, 2020
Summary: Test Plan: Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 2afac9bf2b9b45ed771e613efce63d0ebc944d06 Pull Request resolved: #46738
| self.dequant = DeQuantStub() | ||
|
|
||
| def forward(self, x): | ||
| x = self.quant(x) |
There was a problem hiding this comment.
Does the approach here also work for cases where the scale and zero-point are not fixed? i.e for a hardTanh, the scale and zero-point depend on the arguments to init.
There was a problem hiding this comment.
maybe, this allows to provide special fake quantize(activation_post_process) for a specific pattern (module/functional/torch op), could you write down the details, is it like following?
fake_quant = HardTanhFakeQuantize(hardtanh_instance.params)
I think this should be too hard to support in current implementation.
There was a problem hiding this comment.
Actually for hardtanh, our implementation calls qclamp:
, which doesn't need LUTs. In this case, we should treat it like ReLU, i.e there is no fake-quant or observer needed at the output as there is no quantization that occurs inside the op.There was a problem hiding this comment.
yeah this is how it is implemented right now: https://github.com/pytorch/pytorch/blob/master/torch/quantization/fx/quantization_patterns.py#L512
supriyar
reviewed
Oct 23, 2020
… qat" Summary: Test Plan: Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D24486972](https://our.internmc.facebook.com/intern/diff/D24486972) [ghstack-poisoned]
jerryzh168
added a commit
that referenced
this pull request
Oct 23, 2020
Summary: Test Plan: Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 03ec5b5d9757f7623bca25c8db01b9ea86f4acf9 Pull Request resolved: #46738
… qat" Summary: Test Plan: Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D24486972](https://our.internmc.facebook.com/intern/diff/D24486972) [ghstack-poisoned]
supriyar
reviewed
Oct 23, 2020
supriyar
reviewed
Oct 23, 2020
… qat" Summary: Test Plan: Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D24486972](https://our.internmc.facebook.com/intern/diff/D24486972) [ghstack-poisoned]
… qat" Summary: Test Plan: Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D24486972](https://our.internmc.facebook.com/intern/diff/D24486972) [ghstack-poisoned]
jerryzh168
added a commit
that referenced
this pull request
Oct 23, 2020
Summary: Test Plan: Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: f3435814da778464d9da8ed32771e0fa746bc2ed Pull Request resolved: #46738
raghuramank100
approved these changes
Oct 24, 2020
jerryzh168
added a commit
that referenced
this pull request
Oct 26, 2020
Summary: Test Plan: Imported from OSS Reviewed By: raghuramank100 [ghstack-poisoned]
|
This pull request has been merged in e927b62. |
|
This pull request has been merged in e927b62. |
jerryzh168
added a commit
that referenced
this pull request
Oct 26, 2020
… qat (#46738)" Summary: Test Plan: Imported from OSS Reviewed By: raghuramank100 Differential Revision: [D24547180](https://our.internmc.facebook.com/intern/diff/D24547180) [ghstack-poisoned]
jerryzh168
added a commit
that referenced
this pull request
Oct 26, 2020
Summary: Test Plan: Imported from OSS Reviewed By: raghuramank100 ghstack-source-id: f5d296debbd424a4536ad2c55cfc1646317fdb58 Pull Request resolved: #46871
facebook-github-bot
pushed a commit
that referenced
this pull request
Oct 27, 2020
This was referenced Oct 28, 2020
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Stack from ghstack:
Summary:
Test Plan:
Reviewers:
Subscribers:
Tasks:
Tags:
Differential Revision: D24486972