New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[quant][graphmode][fx] custom_module support static/dynamic/weight_only quant #46786
Conversation
…ly quant Summary: Previously we only support static quant, this PR added support for other types of quantization. Note qat is actually orthogonal to these quant types, this is referring to the convert step where we convert the observed module to a quantized module. for qat, user will provide a CustomModule -> FakeQuantizedCustomModule in prepare_custom_config_dict and FakeQuantizedCustomModule -> static/dynamic/weight_only quantized CustomModule in convert_custom_config_dict. Test Plan: Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned]
…ly quant Summary: Previously we only support static quant, this PR added support for other types of quantization. Note qat is actually orthogonal to these quant types, this is referring to the convert step where we convert the observed module to a quantized module. for qat, user will provide a CustomModule -> FakeQuantizedCustomModule in prepare_custom_config_dict and FakeQuantizedCustomModule -> static/dynamic/weight_only quantized CustomModule in convert_custom_config_dict. Test Plan: Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 05da8385c0669ccbaff62915881b18dd7f52768b Pull Request resolved: #46786
💊 CI failures summary and remediationsAs of commit 6eeb8d7 (more details on the Dr. CI page): 💚 💚 Looks good so far! There are no failures yet. 💚 💚 This comment was automatically generated by Dr. CI (expand for details).Follow this link to opt-out of these comments for your Pull Requests.Please report bugs/suggestions on the GitHub issue tracker or post in the (internal) Dr. CI Users group. This comment has been revised 8 times. |
return quantized | ||
|
||
class DynamicallyQuantizedCustomModule(torch.nn.Module): | ||
def __init__(self, conv): | ||
class DynamicQuantCustomModule(torch.nn.Module): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we have a base class that all DynamicQuantizedCustomModules should inherit from?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why do we need this? to enforce the interface? this might be an improvement in the future if needed i think.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, this is primarily to enforce the interface so that these type of modules have the required from_observed method
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
sure, we can do this in a separate PR
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we simplify the flow for dynamic quant?
…c/weight_only quant" Summary: Previously we only support static quant, this PR added support for other types of quantization. Note qat is actually orthogonal to these quant types, this is referring to the convert step where we convert the observed module to a quantized module. for qat, user will provide a CustomModule -> FakeQuantizedCustomModule in prepare_custom_config_dict and FakeQuantizedCustomModule -> static/dynamic/weight_only quantized CustomModule in convert_custom_config_dict. Test Plan: Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D24514701](https://our.internmc.facebook.com/intern/diff/D24514701) [ghstack-poisoned]
…ly quant Summary: Previously we only support static quant, this PR added support for other types of quantization. Note qat is actually orthogonal to these quant types, this is referring to the convert step where we convert the observed module to a quantized module. for qat, user will provide a CustomModule -> FakeQuantizedCustomModule in prepare_custom_config_dict and FakeQuantizedCustomModule -> static/dynamic/weight_only quantized CustomModule in convert_custom_config_dict. Test Plan: Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 3b5e3a1a2bc03f768dcae0867956e14491fcab76 Pull Request resolved: #46786
…c/weight_only quant" Summary: Previously we only support static quant, this PR added support for other types of quantization. Note qat is actually orthogonal to these quant types, this is referring to the convert step where we convert the observed module to a quantized module. for qat, user will provide a CustomModule -> FakeQuantizedCustomModule in prepare_custom_config_dict and FakeQuantizedCustomModule -> static/dynamic/weight_only quantized CustomModule in convert_custom_config_dict. Test Plan: Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D24514701](https://our.internmc.facebook.com/intern/diff/D24514701) [ghstack-poisoned]
…ly quant Summary: Previously we only support static quant, this PR added support for other types of quantization. Note qat is actually orthogonal to these quant types, this is referring to the convert step where we convert the observed module to a quantized module. for qat, user will provide a CustomModule -> FakeQuantizedCustomModule in prepare_custom_config_dict and FakeQuantizedCustomModule -> static/dynamic/weight_only quantized CustomModule in convert_custom_config_dict. Test Plan: Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 849441a7a8eec52c6510889df3bc1f8597a84546 Pull Request resolved: #46786
This pull request has been merged in 998b9b9. |
Stack from ghstack:
Summary:
Previously we only support static quant, this PR added support for other types of quantization.
Note qat is actually orthogonal to these quant types, this is referring to the convert step where we
convert the observed module to a quantized module.
for qat, user will provide a CustomModule -> FakeQuantizedCustomModule in prepare_custom_config_dict
and FakeQuantizedCustomModule -> static/dynamic/weight_only quantized CustomModule in convert_custom_config_dict.
Test Plan:
Reviewers:
Subscribers:
Tasks:
Tags:
Differential Revision: D24514701