-
Notifications
You must be signed in to change notification settings - Fork 25.7k
Add fake quantize operator that works in backward pass #40532
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
[ghstack-poisoned]
💊 CI failures summary and remediationsAs of commit b9993f4 (more details on the Dr. CI page):
🕵️ 7 new failures recognized by patternsThe following CI failures do not appear to be due to upstream breakages (reran 1 job to discount flakiness):
|
Differential Revision: [D22217029](https://our.internmc.facebook.com/intern/diff/D22217029) [ghstack-poisoned]
Differential Revision: [D22217029](https://our.internmc.facebook.com/intern/diff/D22217029) [ghstack-poisoned]
Differential Revision: [D22217029](https://our.internmc.facebook.com/intern/diff/D22217029) [ghstack-poisoned]
Differential Revision: [D22217029](https://our.internmc.facebook.com/intern/diff/D22217029) [ghstack-poisoned]
Differential Revision: [D22217029](https://our.internmc.facebook.com/intern/diff/D22217029) [ghstack-poisoned]
Differential Revision: [D22217029](https://our.internmc.facebook.com/intern/diff/D22217029) [ghstack-poisoned]
|
Unlanding. This broke multiple OSS CI builds. Example failure: |
ghstack-source-id: 331009d Pull Request resolved: pytorch/pytorch#40532 Adds unit test for validating correctness ghstack-source-id: 331009d Pull Request resolved: pytorch/pytorch#40715
This diff adds
FakeQuantizeWithBackward. This works the same way as the regularFakeQuantizemodule, allowing QAT to occur in the forward pass, except it has an additionalquantize_backwardparameter. Whenquantize_backwardis enabled, the gradients are fake quantized as well (dynamically, using hard-coded values). This allows the user to see whether there would be a significant loss of accuracy if the gradients were quantized in their model.Stack from ghstack:
Differential Revision: D22217029