[ao] Fix to InputWeightEqualization detector to handle Conv groups #81971
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Summary: The current implementation of the InputWeightEqualization detector broke when it was tested on MobileNetV2, and the reason for this is that it wasn't able to properly handle groups in Conv layers, and there also had to be some minor reshaping of the weights to handle this as well. In addition, the output was correspondingly tuned so that instead of giving on output for each channel on each layer, it gives a single suggestion per module and just lets it know how many of the channels could benefit from input-weight equalization, and suggests it if it's more than half. There was also the realization that the test class didn't do a good job of testing different dimensions for the batch vs. height vs. width, so this was updated to be more comprehensive as well. Test Plan: python test/test_quantization TestFxDetectInputWeightEqualization Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned]
🔗 Helpful links
✅ No Failures (0 Pending)As of commit c856e9a (more details on the Dr. CI page): Expand to see more💚 💚 Looks good so far! There are no failures yet. 💚 💚 This comment was automatically generated by Dr. CI (expand for details).Please report bugs/suggestions to the (internal) Dr. CI Users group. |
This was referenced Jul 22, 2022
…v groups" Summary: The current implementation of the InputWeightEqualization detector broke when it was tested on MobileNetV2, and the reason for this is that it wasn't able to properly handle groups in Conv layers, and there also had to be some minor reshaping of the weights to handle this as well. In addition, the output was correspondingly tuned so that instead of giving on output for each channel on each layer, it gives a single suggestion per module and just lets it know how many of the channels could benefit from input-weight equalization, and suggests it if it's more than half. There was also the realization that the test class didn't do a good job of testing different dimensions for the batch vs. height vs. width, so this was updated to be more comprehensive as well. Test Plan: python test/test_quantization TestFxDetectInputWeightEqualization Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned]
jerryzh168
reviewed
Jul 22, 2022
jerryzh168
reviewed
Jul 22, 2022
jerryzh168
reviewed
Jul 22, 2022
jerryzh168
reviewed
Jul 22, 2022
jerryzh168
approved these changes
Jul 22, 2022
…v groups" Summary: The current implementation of the InputWeightEqualization detector broke when it was tested on MobileNetV2, and the reason for this is that it wasn't able to properly handle groups in Conv layers, and there also had to be some minor reshaping of the weights to handle this as well. In addition, the output was correspondingly tuned so that instead of giving on output for each channel on each layer, it gives a single suggestion per module and just lets it know how many of the channels could benefit from input-weight equalization, and suggests it if it's more than half. There was also the realization that the test class didn't do a good job of testing different dimensions for the batch vs. height vs. width, so this was updated to be more comprehensive as well. Test Plan: python test/test_quantization TestFxDetectInputWeightEqualization Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned]
vspenubarthi
added
release notes: quantization
|
@pytorchbot merge -g |
|
@pytorchbot successfully started a merge job. Check the current status here |
facebook-github-bot
pushed a commit
that referenced
this pull request
Jul 26, 2022
…81971) (#81971) Summary: The current implementation of the InputWeightEqualization detector broke when it was tested on MobileNetV2, and the reason for this is that it wasn't able to properly handle groups in Conv layers, and there also had to be some minor reshaping of the weights to handle this as well. In addition, the output was correspondingly tuned so that instead of giving on output for each channel on each layer, it gives a single suggestion per module and just lets it know how many of the channels could benefit from input-weight equalization, and suggests it if it's more than half. There was also the realization that the test class didn't do a good job of testing different dimensions for the batch vs. height vs. width, so this was updated to be more comprehensive as well. Pull Request resolved: #81971 Approved by: https://github.com/jerryzh168 Test Plan: contbuild & OSS CI, see https://hud.pytorch.org/commit/pytorch/pytorch/0cacaf070f4a67f6ef49281be260185e6fac84a3 Test plan from GitHub: python test/test_quantization TestFxDetectInputWeightEqualization Reviewed By: osalpekar Differential Revision: D38119589 Pulled By: vspenubarthi fbshipit-source-id: 3d0a1d2f569cdab0c96eb1d37424366d20d6cf6e
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Stack from ghstack (oldest at bottom):
Summary: The current implementation of the InputWeightEqualization
detector broke when it was tested on MobileNetV2, and the reason for
this is that it wasn't able to properly handle groups in Conv layers,
and there also had to be some minor reshaping of the weights to handle
this as well.
In addition, the output was correspondingly tuned so that instead of
giving on output for each channel on each layer, it gives a single
suggestion per module and just lets it know how many of the channels
could benefit from input-weight equalization, and suggests it if it's
more than half.
There was also the realization that the test class didn't do a good job
of testing different dimensions for the batch vs. height vs. width, so
this was updated to be more comprehensive as well.
Test Plan: python test/test_quantization TestFxDetectInputWeightEqualization
Reviewers:
Subscribers:
Tasks:
Tags: