[quant][pt2e] Support setting qconfig by module_type #92355

jerryzh168 · 2023-01-18T00:00:19Z

Stack from ghstack (oldest at bottom):

-> [quant][pt2e] Support setting qconfig by module_type #92355

Summary:
This PR supports the following feature for QConfigMapping:

qconfig_mapping = QConfigMapping().set_object_type(torch.nn.Conv2d, qconfig)
backend_config = get_qnnpack_pt2e_backend_config()
m = prepare_pt2e(m, qconfig_mapping, example_inputs, backend_config)

which means users want to set the qconfig for all calls to torch.nn.Conv2d to use qconfig, note this is only verified for the case when the module is broken down to a single aten op right now, e.g. torch.nn.Conv2d will be torch.ops.aten.convolution op when traced through. will need to support more complicated modules that is broken down to multiple operators later, e.g. (MaxPool)

Test Plan:
python test/test_quantization.py TestQuantizePT2E.test_qconfig_module_type

Reviewers:

Subscribers:

Tasks:

Tags:

cc @mlazos @soumith @voznesenskym @yanboliang @penguinwu @anijain2305 @EikanWang @jgong5 @Guobing-Chen @chunyuan-w @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @desertfire

Summary: att Test Plan: python test/test_quantization.py TestQuantizePT2E.test_qconfig_module_type Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned]

pytorch-bot · 2023-01-18T00:00:23Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/92355

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit d8ab9cd:
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

Summary: att Test Plan: python test/test_quantization.py TestQuantizePT2E.test_qconfig_module_type Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 18e2c3dd54e788369ad2fcd7df504f901f635aae Pull Request resolved: #92355

jcaip

There's some unused imports the linter is complaining about.

Also does att mean something special for the summary? If not I think you should update with something more descriptive.

jerryzh168 · 2023-01-18T20:00:00Z

There's some unused imports the linter is complaining about.

Also does att mean something special for the summary? If not I think you should update with something more descriptive.

will fix the lint, att means as the title, OK, I added more descriptions

Summary: This PR supports the following feature for QConfigMapping: ``` qconfig_mapping = QConfigMapping().set_object_type(torch.nn.Conv2d, qconfig) backend_config = get_qnnpack_pt2e_backend_config() m = prepare_pt2e(m, qconfig_mapping, example_inputs, backend_config) ``` which means users want to set the qconfig for all calls to `torch.nn.Conv2d` to use `qconfig`, note this is only verified for the case when the module is broken down to a single aten op right now, e.g. torch.nn.Conv2d will be torch.ops.aten.convolution op when traced through. will need to support more complicated modules that is broken down to multiple operators later, e.g. (MaxPool) Test Plan: python test/test_quantization.py TestQuantizePT2E.test_qconfig_module_type Reviewers: Subscribers: Tasks: Tags: cc mlazos soumith voznesenskym yanboliang penguinwu anijain2305 EikanWang jgong5 Guobing-Chen chunyuan-w XiaobingSuper zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx desertfire [ghstack-poisoned]

Summary: att Test Plan: python test/test_quantization.py TestQuantizePT2E.test_qconfig_module_type Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 35a44b0812dbd223f93d2684c7bf1d150cb06ad1 Pull Request resolved: #92355

jcaip

Thanks for adding more info!

Summary: This PR supports the following feature for QConfigMapping: ``` qconfig_mapping = QConfigMapping().set_object_type(torch.nn.Conv2d, qconfig) backend_config = get_qnnpack_pt2e_backend_config() m = prepare_pt2e(m, qconfig_mapping, example_inputs, backend_config) ``` which means users want to set the qconfig for all calls to `torch.nn.Conv2d` to use `qconfig`, note this is only verified for the case when the module is broken down to a single aten op right now, e.g. torch.nn.Conv2d will be torch.ops.aten.convolution op when traced through. will need to support more complicated modules that is broken down to multiple operators later, e.g. (MaxPool) Test Plan: python test/test_quantization.py TestQuantizePT2E.test_qconfig_module_type Reviewers: Subscribers: Tasks: Tags: cc mlazos soumith voznesenskym yanboliang penguinwu anijain2305 EikanWang jgong5 Guobing-Chen chunyuan-w XiaobingSuper zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx desertfire [ghstack-poisoned]

Summary: att Test Plan: python test/test_quantization.py TestQuantizePT2E.test_qconfig_module_type Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 7e1be387d1a715db2c184009ea294bbf75ad7747 Pull Request resolved: #92355

jerryzh168 · 2023-01-19T18:03:45Z

@pytorchbot rebase

pytorchmergebot · 2023-01-19T18:05:59Z

@pytorchbot successfully started a rebase job. Check the current status here

Summary: This PR supports the following feature for QConfigMapping: ``` qconfig_mapping = QConfigMapping().set_object_type(torch.nn.Conv2d, qconfig) backend_config = get_qnnpack_pt2e_backend_config() m = prepare_pt2e(m, qconfig_mapping, example_inputs, backend_config) ``` which means users want to set the qconfig for all calls to `torch.nn.Conv2d` to use `qconfig`, note this is only verified for the case when the module is broken down to a single aten op right now, e.g. torch.nn.Conv2d will be torch.ops.aten.convolution op when traced through. will need to support more complicated modules that is broken down to multiple operators later, e.g. (MaxPool) Test Plan: python test/test_quantization.py TestQuantizePT2E.test_qconfig_module_type Reviewers: Subscribers: Tasks: Tags: cc mlazos soumith voznesenskym yanboliang penguinwu anijain2305 EikanWang jgong5 Guobing-Chen chunyuan-w XiaobingSuper zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx desertfire [ghstack-poisoned]

pytorchmergebot · 2023-01-19T18:06:21Z

Successfully rebased gh/jerryzh168/843/orig onto refs/remotes/origin/viable/strict, please pull locally before adding more changes (for example, via ghstack checkout https://github.com/pytorch/pytorch/pull/92355)

Summary: att Test Plan: python test/test_quantization.py TestQuantizePT2E.test_qconfig_module_type Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: e38d8af9cec207bb3a5e808f70b86c13975d4066 Pull Request resolved: #92355

jerryzh168 · 2023-01-19T23:00:06Z

@pytorchbot merge -g

malfet · 2023-01-19T23:39:22Z

@pytorchbot merge -g

pytorchmergebot · 2023-01-19T23:40:58Z

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

[quant][pt2e] Support setting qconfig by module_type

eed8180

Summary: att Test Plan: python test/test_quantization.py TestQuantizePT2E.test_qconfig_module_type Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned]

pytorch-bot bot added the release notes: quantization release notes category label Jan 18, 2023

github-actions bot added ciflow/inductor module: dynamo labels Jan 18, 2023

jerryzh168 requested review from SherlockNoMad, vkuzo, jcaip and andrewor14 January 18, 2023 00:01

jcaip reviewed Jan 18, 2023

View reviewed changes

jerryzh168 requested a review from jcaip January 18, 2023 21:57

jcaip approved these changes Jan 18, 2023

View reviewed changes

pytorch-bot bot added the ciflow/trunk Trigger trunk jobs on your pull request label Jan 19, 2023

malfet added the topic: improvements topic category label Jan 19, 2023

pytorchmergebot added the Merged label Jan 20, 2023

pytorchmergebot closed this in 1464db0 Jan 20, 2023

facebook-github-bot deleted the gh/jerryzh168/843/head branch June 8, 2023 17:38

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[quant][pt2e] Support setting qconfig by module_type #92355

[quant][pt2e] Support setting qconfig by module_type #92355

jerryzh168 commented Jan 18, 2023 •

edited

pytorch-bot bot commented Jan 18, 2023 •

edited

jcaip left a comment

jerryzh168 commented Jan 18, 2023 •

edited

jcaip left a comment

jerryzh168 commented Jan 19, 2023

pytorchmergebot commented Jan 19, 2023

pytorchmergebot commented Jan 19, 2023

jerryzh168 commented Jan 19, 2023

malfet commented Jan 19, 2023

pytorchmergebot commented Jan 19, 2023

[quant][pt2e] Support setting qconfig by module_type #92355

[quant][pt2e] Support setting qconfig by module_type #92355

Conversation

jerryzh168 commented Jan 18, 2023 • edited

pytorch-bot bot commented Jan 18, 2023 • edited

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/92355

✅ No Failures

jcaip left a comment

Choose a reason for hiding this comment

jerryzh168 commented Jan 18, 2023 • edited

jcaip left a comment

Choose a reason for hiding this comment

jerryzh168 commented Jan 19, 2023

pytorchmergebot commented Jan 19, 2023

pytorchmergebot commented Jan 19, 2023

jerryzh168 commented Jan 19, 2023

malfet commented Jan 19, 2023

pytorchmergebot commented Jan 19, 2023

Merge started

jerryzh168 commented Jan 18, 2023 •

edited

pytorch-bot bot commented Jan 18, 2023 •

edited

jerryzh168 commented Jan 18, 2023 •

edited