[ao] Added Equalization QConfig generation to ModelReport class #83698

vspenubarthi · 2022-08-18T19:37:34Z

Stack from ghstack (oldest at bottom):

-> [ao] Added Equalization QConfig generation to ModelReport class #83698

Summary: This adds the capability to generate a QConfigMapping based on
the suggestions of the ModelReport API for the user to use. The only
dependency of this feature is that the calibration is run before the
generation of the QConfigMapping and there is no dependency on the
report generation other than that the observers cannot be removed before
this is called. This maps module fqns to EqualizationQConfigs instead of regular
QConfigs.

Example Usage (after callibration):

quantization_mapping = mod_report.generate_qconfig_mapping()
equalization_mapping = mod_report.generate_equalization_mapping()

prepared_model = quantize_fx.prepare_fx(model, mapping, example_input, _equalization_config=equalization_mapping)

quantized_model = quantize_fx.convert_fx(prepared)

This was tested by ensuring that the suggestions generated in the QConfigMapping are:
1. Correct according to the set backend and data passed through
2. Able to be prepared and converted as a proper config (is a valid config)
The test for this is a part of the TestFxModelReportClass test suite.

Test Plan: python test/test_quantization.py TestFxModelReportClass.test_equalization_mapping_generation

Reviewers:

Subscribers:

Tasks:

Tags:

Summary: This adds the capability to generate a QConfigMapping based on the suggestions of the ModelReport API for the user to use. The only dependency of this feature is that the calibration is run before the generation of the QConfigMapping and there is no dependency on the report generation other than that the observers cannot be removed before this is called. This maps module fqns to EqualizationQConfigs instead of regular QConfigs. Example Usage (after callibration): ``` quantization_mapping = mod_report.generate_qconfig_mapping() equalization_mapping = mod_report.generate_equalization_mapping() prepared_model = quantize_fx.prepare_fx(model, mapping, example_input, _equalization_config=equalization_mapping) quantized_model = quantize_fx.convert_fx(prepared) ``` This was tested by ensuring that the suggestions generated in the QConfigMapping are: 1. Correct according to the set backend and data passed through 2. Able to be prepared and converted as a proper config (is a valid config) The test for this is a part of the TestFxModelReportClass test suite. Test Plan: python test/test_quantization.py TestFxModelReportClass.test_equalization_mapping_generation Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned]

facebook-github-bot · 2022-08-18T19:37:41Z

🔗 Helpful links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/83698
📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓Need help or want to give feedback on the CI? Visit our office hours

✅ No Failures (0 Pending)

As of commit 0122b5b (more details on the Dr. CI page):

Expand to see more

💚 💚 Looks good so far! There are no failures yet. 💚 💚

This comment was automatically generated by Dr. CI (expand for details).

Please report bugs/suggestions to the (internal) Dr. CI Users group.

Click here to manually regenerate this comment.

…class" Summary: This adds the capability to generate a QConfigMapping based on the suggestions of the ModelReport API for the user to use. The only dependency of this feature is that the calibration is run before the generation of the QConfigMapping and there is no dependency on the report generation other than that the observers cannot be removed before this is called. This maps module fqns to EqualizationQConfigs instead of regular QConfigs. Example Usage (after callibration): ``` quantization_mapping = mod_report.generate_qconfig_mapping() equalization_mapping = mod_report.generate_equalization_mapping() prepared_model = quantize_fx.prepare_fx(model, mapping, example_input, _equalization_config=equalization_mapping) quantized_model = quantize_fx.convert_fx(prepared) ``` This was tested by ensuring that the suggestions generated in the QConfigMapping are: 1. Correct according to the set backend and data passed through 2. Able to be prepared and converted as a proper config (is a valid config) The test for this is a part of the TestFxModelReportClass test suite. Test Plan: python test/test_quantization.py TestFxModelReportClass.test_equalization_mapping_generation Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned]

torch/ao/quantization/fx/_model_report/model_report.py

…class" Summary: This adds the capability to generate a QConfigMapping based on the suggestions of the ModelReport API for the user to use. The only dependency of this feature is that the calibration is run before the generation of the QConfigMapping and there is no dependency on the report generation other than that the observers cannot be removed before this is called. This maps module fqns to EqualizationQConfigs instead of regular QConfigs. Example Usage (after callibration): ``` quantization_mapping = mod_report.generate_qconfig_mapping() equalization_mapping = mod_report.generate_equalization_mapping() prepared_model = quantize_fx.prepare_fx(model, mapping, example_input, _equalization_config=equalization_mapping) quantized_model = quantize_fx.convert_fx(prepared) ``` This was tested by ensuring that the suggestions generated in the QConfigMapping are: 1. Correct according to the set backend and data passed through 2. Able to be prepared and converted as a proper config (is a valid config) The test for this is a part of the TestFxModelReportClass test suite. Test Plan: python test/test_quantization.py TestFxModelReportClass.test_equalization_mapping_generation Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned]

jerryzh168

looks good

jerryzh168 · 2022-08-18T20:23:07Z

torch/ao/quantization/fx/_model_report/model_report.py

        # return the generated mapping
        return mapping

+    def _quantization_config_generator(self, detector_qconfig_info: DetectorQConfigInfo, module: torch.nn.Module) -> QConfig:


I feel ideally we'd like to remove these one-line functions

I understand that helper functions improves code reuse, but creating too many helper functions will also make the code less readable

vspenubarthi · 2022-08-18T20:25:46Z

@pytorchbot merge -g

pytorchmergebot · 2022-08-18T20:27:12Z

@pytorchbot successfully started a merge job. Check the current status here.
The merge job was triggered with the green (-g) flag. This means that your change will be merged once all checks on your PR have passed (ETA: 0-4 Hours). If this is not the intended behavior, feel free to use some of the other merge options in the wiki.
Please reach out to the PyTorch DevX Team with feedback or questions!

pytorchmergebot · 2022-08-18T23:28:02Z

Merge failed
Reason: Command git -C /home/runner/work/pytorch/pytorch cherry-pick -x f2b1cc88a0b27740bd33cb37ea29cc4919ad6977 returned non-zero exit code 1

Auto-merging test/quantization/fx/test_model_report_fx.py
Auto-merging torch/ao/quantization/fx/_model_report/detector.py
CONFLICT (content): Merge conflict in torch/ao/quantization/fx/_model_report/detector.py
Auto-merging torch/ao/quantization/fx/_model_report/model_report.py
CONFLICT (content): Merge conflict in torch/ao/quantization/fx/_model_report/model_report.py
error: could not apply f2b1cc88a0... [ao] Added Equalization QConfig generation to ModelReport class
hint: After resolving the conflicts, mark them with
hint: "git add/rm <pathspec>", then run
hint: "git cherry-pick --continue".
hint: You can instead skip this commit with "git cherry-pick --skip".
hint: To abort and get back to the state before "git cherry-pick",
hint: run "git cherry-pick --abort".

Raised by https://github.com/pytorch/pytorch/actions/runs/2885197830

…class" Summary: This adds the capability to generate a QConfigMapping based on the suggestions of the ModelReport API for the user to use. The only dependency of this feature is that the calibration is run before the generation of the QConfigMapping and there is no dependency on the report generation other than that the observers cannot be removed before this is called. This maps module fqns to EqualizationQConfigs instead of regular QConfigs. Example Usage (after callibration): ``` quantization_mapping = mod_report.generate_qconfig_mapping() equalization_mapping = mod_report.generate_equalization_mapping() prepared_model = quantize_fx.prepare_fx(model, mapping, example_input, _equalization_config=equalization_mapping) quantized_model = quantize_fx.convert_fx(prepared) ``` This was tested by ensuring that the suggestions generated in the QConfigMapping are: 1. Correct according to the set backend and data passed through 2. Able to be prepared and converted as a proper config (is a valid config) The test for this is a part of the TestFxModelReportClass test suite. Test Plan: python test/test_quantization.py TestFxModelReportClass.test_equalization_mapping_generation Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned]

Summary: This adds the capability to generate a QConfigMapping based on the suggestions of the ModelReport API for the user to use. The only dependency of this feature is that the calibration is run before the generation of the QConfigMapping and there is no dependency on the report generation other than that the observers cannot be removed before this is called. This maps module fqns to EqualizationQConfigs instead of regular QConfigs. Example Usage (after callibration): ``` quantization_mapping = mod_report.generate_qconfig_mapping() equalization_mapping = mod_report.generate_equalization_mapping() prepared_model = quantize_fx.prepare_fx(model, mapping, example_input, _equalization_config=equalization_mapping) quantized_model = quantize_fx.convert_fx(prepared) ``` This was tested by ensuring that the suggestions generated in the QConfigMapping are: 1. Correct according to the set backend and data passed through 2. Able to be prepared and converted as a proper config (is a valid config) The test for this is a part of the TestFxModelReportClass test suite. Test Plan: python test/test_quantization.py TestFxModelReportClass.test_equalization_mapping_generation Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 5d4c9dc Pull Request resolved: #83698

vspenubarthi · 2022-08-18T23:42:51Z

@pytorchbot merge -g

pytorchmergebot · 2022-08-18T23:45:13Z

@pytorchbot successfully started a merge job. Check the current status here.
The merge job was triggered with the green (-g) flag. This means that your change will be merged once all checks on your PR have passed (ETA: 0-4 Hours). If this is not the intended behavior, feel free to use some of the other merge options in the wiki.
Please reach out to the PyTorch DevX Team with feedback or questions!

…) (#83698) Summary: This adds the capability to generate a QConfigMapping based on the suggestions of the ModelReport API for the user to use. The only dependency of this feature is that the calibration is run before the generation of the QConfigMapping and there is no dependency on the report generation other than that the observers cannot be removed before this is called. This maps module fqns to EqualizationQConfigs instead of regular QConfigs. Example Usage (after callibration): ``` quantization_mapping = mod_report.generate_qconfig_mapping() equalization_mapping = mod_report.generate_equalization_mapping() prepared_model = quantize_fx.prepare_fx(model, mapping, example_input, _equalization_config=equalization_mapping) quantized_model = quantize_fx.convert_fx(prepared) ``` This was tested by ensuring that the suggestions generated in the QConfigMapping are: 1. Correct according to the set backend and data passed through 2. Able to be prepared and converted as a proper config (is a valid config) The test for this is a part of the TestFxModelReportClass test suite. Pull Request resolved: #83698 Approved by: https://github.com/jerryzh168 Test Plan: contbuild & OSS CI, see https://hud.pytorch.org/commit/pytorch/pytorch/88e0165d085166ce13ef443991eea003ee86869e Test plan from GitHub: python test/test_quantization.py TestFxModelReportClass.test_equalization_mapping_generation Reviewed By: atalman Differential Revision: D38853641 Pulled By: vspenubarthi fbshipit-source-id: a7f365d62e1e0fbb74b051ef2d3e1bc4b8cf04e2

vspenubarthi mentioned this pull request Aug 18, 2022

[ao] Added Quantization QConfig generation to ModelReport class #83688

Closed

facebook-github-bot added the cla signed label Aug 18, 2022

facebook-github-bot added the fx label Aug 18, 2022

vspenubarthi added release notes: quantization release notes category topic: new features topic category labels Aug 18, 2022

jerryzh168 reviewed Aug 18, 2022

View reviewed changes

torch/ao/quantization/fx/_model_report/model_report.py Show resolved Hide resolved

jerryzh168 reviewed Aug 18, 2022

View reviewed changes

torch/ao/quantization/fx/_model_report/model_report.py Outdated Show resolved Hide resolved

jerryzh168 reviewed Aug 18, 2022

View reviewed changes

torch/ao/quantization/fx/_model_report/model_report.py Show resolved Hide resolved

jerryzh168 approved these changes Aug 18, 2022

View reviewed changes

jerryzh168 reviewed Aug 18, 2022

View reviewed changes

pytorchmergebot added the Merged label Aug 19, 2022

pytorchmergebot closed this in 88e0165 Aug 19, 2022

facebook-github-bot deleted the gh/vspenubarthi/16/head branch August 22, 2022 14:19

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[ao] Added Equalization QConfig generation to ModelReport class #83698

[ao] Added Equalization QConfig generation to ModelReport class #83698

Uh oh!

vspenubarthi commented Aug 18, 2022 •

edited

Loading

Uh oh!

facebook-github-bot commented Aug 18, 2022 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jerryzh168 left a comment

Uh oh!

jerryzh168 Aug 18, 2022

Uh oh!

jerryzh168 Aug 18, 2022 •

edited

Loading

Uh oh!

vspenubarthi commented Aug 18, 2022

Uh oh!

pytorchmergebot commented Aug 18, 2022

Uh oh!

pytorchmergebot commented Aug 18, 2022

Uh oh!

vspenubarthi commented Aug 18, 2022

Uh oh!

pytorchmergebot commented Aug 18, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

[ao] Added Equalization QConfig generation to ModelReport class #83698

[ao] Added Equalization QConfig generation to ModelReport class #83698

Uh oh!

Conversation

vspenubarthi commented Aug 18, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

facebook-github-bot commented Aug 18, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful links

✅ No Failures (0 Pending)

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jerryzh168 left a comment

Choose a reason for hiding this comment

Uh oh!

jerryzh168 Aug 18, 2022

Choose a reason for hiding this comment

Uh oh!

jerryzh168 Aug 18, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

vspenubarthi commented Aug 18, 2022

Uh oh!

pytorchmergebot commented Aug 18, 2022

Uh oh!

pytorchmergebot commented Aug 18, 2022

Uh oh!

vspenubarthi commented Aug 18, 2022

Uh oh!

pytorchmergebot commented Aug 18, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

vspenubarthi commented Aug 18, 2022 •

edited

Loading

facebook-github-bot commented Aug 18, 2022 •

edited

Loading

jerryzh168 Aug 18, 2022 •

edited

Loading