-
Notifications
You must be signed in to change notification settings - Fork 21.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[quant][graphmode][fx] Change standalone module api #49719
Conversation
Summary: We find there are multiple use cases for standalone module, one use case requires standalone module to produce a module that takes float Tensor as input and outputs a float Tensor, the other needs to produce a modulee that takes quantized Tensor as input and outputs a quantized Tensor. This is similar to `quantized_input_idxs` and `quantized_output_idxs` so we want to nest prepare_custom_config_dict in the standalone module configuration, for maximum flxibility we also include qconfig_dict for stand alone module as well in case user needs to have special qconfig_dict for the standalone module in the future. Changed from ```python prepare_custom_config_dict = { "standalone_module_name": ["standalone_module"], "standalone_module_class": [StandaloneModule] } ``` to ```python prepare_custom_config_dict = { "standalone_module_name": [("standalone_module", qconfig_dict1, prepare_custom_config_dict1)], "standalone_module_class": [(StandaloneModule, qconfig_dict2, prepare_custom_config_dict2)] } ``` The entries in the config are: 1. name/module_class 2. optional qconfig_dict, when it is None, we'll use {"": qconfig} where qconfig is the one from parent qconfig_dict 3. optional prepare_custom_config_dict, when it is None, we'll use default value of prepare_custom_config_dict for prepare API (None) Test Plan: python test/test_quantization.py TestQuantizeFx.test_standalone_module Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned]
💊 CI failures summary and remediationsAs of commit 07eab29 (more details on the Dr. CI page):
This comment was automatically generated by Dr. CI (expand for details).Follow this link to opt-out of these comments for your Pull Requests.Please report bugs/suggestions to the (internal) Dr. CI Users group. This comment has been revised 10 times. |
Summary: We find there are multiple use cases for standalone module, one use case requires standalone module to produce a module that takes float Tensor as input and outputs a float Tensor, the other needs to produce a modulee that takes quantized Tensor as input and outputs a quantized Tensor. This is similar to `quantized_input_idxs` and `quantized_output_idxs` so we want to nest prepare_custom_config_dict in the standalone module configuration, for maximum flxibility we also include qconfig_dict for stand alone module as well in case user needs to have special qconfig_dict for the standalone module in the future. Changed from ```python prepare_custom_config_dict = { "standalone_module_name": ["standalone_module"], "standalone_module_class": [StandaloneModule] } ``` to ```python prepare_custom_config_dict = { "standalone_module_name": [("standalone_module", qconfig_dict1, prepare_custom_config_dict1)], "standalone_module_class": [(StandaloneModule, qconfig_dict2, prepare_custom_config_dict2)] } ``` The entries in the config are: 1. name/module_class 2. optional qconfig_dict, when it is None, we'll use {"": qconfig} where qconfig is the one from parent qconfig_dict 3. optional prepare_custom_config_dict, when it is None, we'll use default value of prepare_custom_config_dict for prepare API (None) Test Plan: python test/test_quantization.py TestQuantizeFx.test_standalone_module Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: c5c100bd93256c96dc8dcdf9664a92d547e47403 Pull Request resolved: #49719
torch/quantization/quantize_fx.py
Outdated
@@ -178,11 +178,13 @@ def prepare_fx( | |||
# optional: specify the path for standalone modules | |||
# These modules are symbolically traced and quantized as one unit | |||
"standalone_module_name": [ | |||
"submodule.standalone" | |||
# module_name, qconfig_dict, prepare_custom_config_dict | |||
("submodule.standalone", None, None) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
does qconfig_dict
== None here means "do not quantize" or "use parent qconfig"?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
it means "use parent qconfig"
for "do not quantize": qconfit_dict = {"": None}
@@ -575,8 +575,8 @@ def forward(self, x): | |||
original_ref_m.conv2.bias = torch.nn.Parameter(original_m.standalone.conv.bias.detach()) | |||
|
|||
qconfig_dict = {"": default_qconfig} | |||
config_name = {"standalone_module_name": ["standalone"]} | |||
config_class = {"standalone_module_class": [StandaloneModule]} | |||
config_name = {"standalone_module_name": [("standalone", None, None)]} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we also test for selective quantization of inputs/outputs for standalone modules?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yes, it's in the next PR
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry, just saw that
Summary: We find there are multiple use cases for standalone module, one use case requires standalone module to produce a module that takes float Tensor as input and outputs a float Tensor, the other needs to produce a modulee that takes quantized Tensor as input and outputs a quantized Tensor. This is similar to `quantized_input_idxs` and `quantized_output_idxs` so we want to nest prepare_custom_config_dict in the standalone module configuration, for maximum flxibility we also include qconfig_dict for stand alone module as well in case user needs to have special qconfig_dict for the standalone module in the future. Changed from ```python prepare_custom_config_dict = { "standalone_module_name": ["standalone_module"], "standalone_module_class": [StandaloneModule] } ``` to ```python prepare_custom_config_dict = { "standalone_module_name": [("standalone_module", qconfig_dict1, prepare_custom_config_dict1)], "standalone_module_class": [(StandaloneModule, qconfig_dict2, prepare_custom_config_dict2)] } ``` The entries in the config are: 1. name/module_class 2. optional qconfig_dict, when it is None, we'll use {"": qconfig} where qconfig is the one from parent qconfig_dict 3. optional prepare_custom_config_dict, when it is None, we'll use default value of prepare_custom_config_dict for prepare API (None) Test Plan: python test/test_quantization.py TestQuantizeFx.test_standalone_module Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D25675704](https://our.internmc.facebook.com/intern/diff/D25675704) [ghstack-poisoned]
Codecov Report
@@ Coverage Diff @@
## gh/jerryzh168/525/base #49719 +/- ##
==========================================================
- Coverage 80.67% 80.58% -0.09%
==========================================================
Files 1895 1895
Lines 205307 205316 +9
==========================================================
- Hits 165622 165446 -176
- Misses 39685 39870 +185 |
This pull request has been merged in f474ffa. |
Stack from ghstack:
Summary:
We find there are multiple use cases for standalone module, one use case requires standalone module
to produce a module that takes float Tensor as input and outputs a float Tensor, the other needs to
produce a modulee that takes quantized Tensor as input and outputs a quantized Tensor.
This is similar to
quantized_input_idxs
andquantized_output_idxs
so we want to nestprepare_custom_config_dict in the standalone module configuration, for maximum flxibility we also
include qconfig_dict for stand alone module as well in case user needs to have special qconfig_dict for
the standalone module in the future.
Changed from
to
The entries in the config are:
Test Plan:
python test/test_quantization.py TestQuantizeFx.test_standalone_module
Reviewers:
Subscribers:
Tasks:
Tags:
Differential Revision: D25675704