New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[quant][pt2e] Support conv bn fusion in convert step for QAT flow #100442

Closed

jerryzh168 wants to merge 1 commit into pytorch:main from jerryzh168:export-D45344281

Contributor

jerryzh168 commented May 2, 2023 •

edited by pytorch-bot bot

Summary:
This PR adds support for folding bn weights into conv for QAT flow, this is equivalent
to the QAT branch of from_float in eager mode quantized conv module: https://github.com/pytorch/pytorch/blob/main/torch/ao/nn/quantized/modules/conv.py#L223

Items that needs followup:

there are some workaround I did because quantize_per_tensor is using float/int args and dynamo does not support these args, need to fix after we change the quantized model representation and also change these args to Tensor

Test Plan: buck2 test @//mode/opt //caffe2/test:quantization_pt2e -- --exact 'caffe2/test:quantization_pt2e - test_convert_qat_conv_bn_fusion (quantization.pt2e.test_quantize_pt2e.TestQuantizePT2E)'

Reviewed By: andrewor14

Differential Revision: D45344281

cc @soumith @voznesenskym @penguinwu @anijain2305 @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @Xia-Weiwen @wenzhe-nrv @jiayisunx @desertfire

pytorch-bot bot added the release notes: quantization label

pytorch-bot bot commented May 2, 2023 •

edited

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/100442

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 25f6957:
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

github-actions bot added ciflow/inductor module: dynamo labels

facebook-github-bot added the fb-exported label

Contributor

facebook-github-bot commented May 2, 2023

This pull request was exported from Phabricator. Differential Revision: D45344281

jerryzh168 force-pushed the export-D45344281 branch from 97aaa4c to 2ab6227 Compare

May 3, 2023 17:47

Contributor

facebook-github-bot commented May 3, 2023

This pull request was exported from Phabricator. Differential Revision: D45344281

jerryzh168 added a commit to jerryzh168/pytorch that referenced this pull request


          [quant][pt2e] Support conv bn fusion in convert step for QAT flow (py…

2ab6227

…torch#100442)

Summary:
Pull Request resolved: pytorch#100442

This PR adds support for folding bn weights into conv for QAT flow, this is equivalent
to the QAT branch of `from_float` in eager mode quantized conv module: https://github.com/pytorch/pytorch/blob/main/torch/ao/nn/quantized/modules/conv.py#L223

Items that needs followup:
* there are some workaround I did because quantize_per_tensor is using float/int args and dynamo does not support these args, need to fix after we change the quantized model representation and also change these args to Tensor

Test Plan: buck2 test @//mode/opt //caffe2/test:quantization_pt2e -- --exact 'caffe2/test:quantization_pt2e - test_convert_qat_conv_bn_fusion (quantization.pt2e.test_quantize_pt2e.TestQuantizePT2E)'

Differential Revision: D45344281

fbshipit-source-id: 36a85a4ba990c3c533f334c8b9f9b3baa9f501ca

Contributor

facebook-github-bot commented May 3, 2023

This pull request was exported from Phabricator. Differential Revision: D45344281

jerryzh168 added a commit to jerryzh168/pytorch that referenced this pull request


          [quant][pt2e] Support conv bn fusion in convert step for QAT flow (py…

eb07ccc

…torch#100442)

Summary:
Pull Request resolved: pytorch#100442

This PR adds support for folding bn weights into conv for QAT flow, this is equivalent
to the QAT branch of `from_float` in eager mode quantized conv module: https://github.com/pytorch/pytorch/blob/main/torch/ao/nn/quantized/modules/conv.py#L223

Items that needs followup:
* there is a workaround that removes overload (.Tensor) for q/dq ops for the match pattern graph that we get from torchdynamo export, we can remove it after we change the quantized model representation

Test Plan: buck2 test @//mode/opt //caffe2/test:quantization_pt2e -- --exact 'caffe2/test:quantization_pt2e - test_convert_qat_conv_bn_fusion (quantization.pt2e.test_quantize_pt2e.TestQuantizePT2E)'

Differential Revision: D45344281

fbshipit-source-id: 92d4a6694c91ccce5050202b367daf40c6cef6d9

jerryzh168 force-pushed the export-D45344281 branch 2 times, most recently from eb07ccc to e3887f2 Compare

May 4, 2023 00:54

jerryzh168 added a commit to jerryzh168/pytorch that referenced this pull request


          [quant][pt2e] Support conv bn fusion in convert step for QAT flow (py…

e3887f2

…torch#100442)

Summary:
Pull Request resolved: pytorch#100442

This PR adds support for folding bn weights into conv for QAT flow, this is equivalent
to the QAT branch of `from_float` in eager mode quantized conv module: https://github.com/pytorch/pytorch/blob/main/torch/ao/nn/quantized/modules/conv.py#L223

Items that needs followup:
* there is a workaround that removes overload (.Tensor) for q/dq ops for the match pattern graph that we get from torchdynamo export, we can remove it after we change the quantized model representation

Test Plan: buck2 test @//mode/opt //caffe2/test:quantization_pt2e -- --exact 'caffe2/test:quantization_pt2e - test_convert_qat_conv_bn_fusion (quantization.pt2e.test_quantize_pt2e.TestQuantizePT2E)'

Differential Revision: D45344281

fbshipit-source-id: 942ff42df3a66ed5a8d408dc3da791b567286c9d

Contributor

facebook-github-bot commented May 4, 2023

This pull request was exported from Phabricator. Differential Revision: D45344281

jerryzh168 added a commit to jerryzh168/pytorch that referenced this pull request


          [quant][pt2e] Support conv bn fusion in convert step for QAT flow (py…

03cc8b0

…torch#100442)

Summary:
Pull Request resolved: pytorch#100442

This PR adds support for folding bn weights into conv for QAT flow, this is equivalent
to the QAT branch of `from_float` in eager mode quantized conv module: https://github.com/pytorch/pytorch/blob/main/torch/ao/nn/quantized/modules/conv.py#L223

Items that needs followup:
* there is a workaround that removes overload (.Tensor) for q/dq ops for the match pattern graph that we get from torchdynamo export, we can remove it after we change the quantized model representation

Test Plan: buck2 test @//mode/opt //caffe2/test:quantization_pt2e -- --exact 'caffe2/test:quantization_pt2e - test_convert_qat_conv_bn_fusion (quantization.pt2e.test_quantize_pt2e.TestQuantizePT2E)'

Differential Revision: D45344281

fbshipit-source-id: 93955601ece05f7286b1dbfb3285f6ff460162d5

jerryzh168 force-pushed the export-D45344281 branch from e3887f2 to 03cc8b0 Compare

May 4, 2023 01:03

Contributor

facebook-github-bot commented May 4, 2023

This pull request was exported from Phabricator. Differential Revision: D45344281

kimishpatel approved these changes

View reviewed changes

jerryzh168 force-pushed the export-D45344281 branch from 03cc8b0 to a1dd1dc Compare

May 8, 2023 21:46

Contributor

facebook-github-bot commented May 8, 2023

This pull request was exported from Phabricator. Differential Revision: D45344281

jerryzh168 added a commit to jerryzh168/pytorch that referenced this pull request


          [quant][pt2e] Support conv bn fusion in convert step for QAT flow (py…

a1dd1dc

…torch#100442)

Summary:
Pull Request resolved: pytorch#100442

This PR adds support for folding bn weights into conv for QAT flow, this is equivalent
to the QAT branch of `from_float` in eager mode quantized conv module: https://github.com/pytorch/pytorch/blob/main/torch/ao/nn/quantized/modules/conv.py#L223

Items that needs followup:
* there is a workaround that removes overload (.Tensor) for q/dq ops for the match pattern graph that we get from torchdynamo export, we can remove it after we change the quantized model representation

Test Plan: buck2 test @//mode/opt //caffe2/test:quantization_pt2e -- --exact 'caffe2/test:quantization_pt2e - test_convert_qat_conv_bn_fusion (quantization.pt2e.test_quantize_pt2e.TestQuantizePT2E)'

Reviewed By: kimishpatel

Differential Revision: D45344281

fbshipit-source-id: ea11bd8187325031aaaace9a6368bac8e44d28d7

jerryzh168 added a commit to jerryzh168/pytorch that referenced this pull request


          [quant][pt2e] Support conv bn fusion in convert step for QAT flow (py…

fc44b44

…torch#100442)

Summary:
Pull Request resolved: pytorch#100442

This PR adds support for folding bn weights into conv for QAT flow, this is equivalent
to the QAT branch of `from_float` in eager mode quantized conv module: https://github.com/pytorch/pytorch/blob/main/torch/ao/nn/quantized/modules/conv.py#L223

Items that needs followup:
* there is a workaround that removes overload (.Tensor) for q/dq ops for the match pattern graph that we get from torchdynamo export, we can remove it after we change the quantized model representation

Test Plan: buck2 test @//mode/opt //caffe2/test:quantization_pt2e -- --exact 'caffe2/test:quantization_pt2e - test_convert_qat_conv_bn_fusion (quantization.pt2e.test_quantize_pt2e.TestQuantizePT2E)'

Reviewed By: kimishpatel

Differential Revision: D45344281

fbshipit-source-id: 55252875f4189d9f26154c528b2d03d19808f2d6

jerryzh168 force-pushed the export-D45344281 branch from a1dd1dc to fc44b44 Compare

May 8, 2023 22:13

Contributor

facebook-github-bot commented May 8, 2023

This pull request was exported from Phabricator. Differential Revision: D45344281

1 similar comment

Contributor

facebook-github-bot commented May 8, 2023

This pull request was exported from Phabricator. Differential Revision: D45344281

jerryzh168 added a commit to jerryzh168/pytorch that referenced this pull request


          [quant][pt2e] Support conv bn fusion in convert step for QAT flow (py…

f56c83d

…torch#100442)

Summary:
Pull Request resolved: pytorch#100442

This PR adds support for folding bn weights into conv for QAT flow, this is equivalent
to the QAT branch of `from_float` in eager mode quantized conv module: https://github.com/pytorch/pytorch/blob/main/torch/ao/nn/quantized/modules/conv.py#L223

Items that needs followup:
* there is a workaround that removes overload (.Tensor) for q/dq ops for the match pattern graph that we get from torchdynamo export, we can remove it after we change the quantized model representation

Test Plan: buck2 test @//mode/opt //caffe2/test:quantization_pt2e -- --exact 'caffe2/test:quantization_pt2e - test_convert_qat_conv_bn_fusion (quantization.pt2e.test_quantize_pt2e.TestQuantizePT2E)'

Reviewed By: kimishpatel

Differential Revision: D45344281

fbshipit-source-id: 5a8988a70a6bc71a608a780bda78b105c2843963

jerryzh168 force-pushed the export-D45344281 branch from fc44b44 to f56c83d Compare

May 8, 2023 23:08

Contributor

facebook-github-bot commented May 8, 2023

This pull request was exported from Phabricator. Differential Revision: D45344281

jerryzh168 added a commit to jerryzh168/pytorch that referenced this pull request


          [quant][pt2e] Support conv bn fusion in convert step for QAT flow (py…

334df3c

…torch#100442)

Summary:
Pull Request resolved: pytorch#100442

This PR adds support for folding bn weights into conv for QAT flow, this is equivalent
to the QAT branch of `from_float` in eager mode quantized conv module: https://github.com/pytorch/pytorch/blob/main/torch/ao/nn/quantized/modules/conv.py#L223

Items that needs followup:
* there is a workaround that removes overload (.Tensor) for q/dq ops for the match pattern graph that we get from torchdynamo export, we can remove it after we change the quantized model representation

Test Plan: buck2 test @//mode/opt //caffe2/test:quantization_pt2e -- --exact 'caffe2/test:quantization_pt2e - test_convert_qat_conv_bn_fusion (quantization.pt2e.test_quantize_pt2e.TestQuantizePT2E)'

Reviewed By: kimishpatel

Differential Revision: D45344281

fbshipit-source-id: d0369f7d5e23a7b30d4cbee55a04394ba106f55f

jerryzh168 force-pushed the export-D45344281 branch from f56c83d to 334df3c Compare

May 8, 2023 23:16

jerryzh168 added a commit to jerryzh168/pytorch that referenced this pull request


          [quant][pt2e] Support conv bn fusion in convert step for QAT flow (py…

f162b5a

…torch#100442)

Summary:
Pull Request resolved: pytorch#100442

This PR adds support for folding bn weights into conv for QAT flow, this is equivalent
to the QAT branch of `from_float` in eager mode quantized conv module: https://github.com/pytorch/pytorch/blob/main/torch/ao/nn/quantized/modules/conv.py#L223

Items that needs followup:
* there is a workaround that removes overload (.Tensor) for q/dq ops for the match pattern graph that we get from torchdynamo export, we can remove it after we change the quantized model representation

Test Plan: buck2 test @//mode/opt //caffe2/test:quantization_pt2e -- --exact 'caffe2/test:quantization_pt2e - test_convert_qat_conv_bn_fusion (quantization.pt2e.test_quantize_pt2e.TestQuantizePT2E)'

Reviewed By: kimishpatel

Differential Revision: D45344281

fbshipit-source-id: 5381416f9b16d2cac4976d168d25210fe2dfdc1f

jerryzh168 force-pushed the export-D45344281 branch from 334df3c to f162b5a Compare

May 8, 2023 23:45

Contributor

facebook-github-bot commented May 8, 2023

This pull request was exported from Phabricator. Differential Revision: D45344281

1 similar comment

Contributor

facebook-github-bot commented May 8, 2023

This pull request was exported from Phabricator. Differential Revision: D45344281

jerryzh168 added a commit to jerryzh168/pytorch that referenced this pull request


          [quant][pt2e] Support conv bn fusion in convert step for QAT flow (py…

f183868

…torch#100442)

Summary:
Pull Request resolved: pytorch#100442

This PR adds support for folding bn weights into conv for QAT flow, this is equivalent
to the QAT branch of `from_float` in eager mode quantized conv module: https://github.com/pytorch/pytorch/blob/main/torch/ao/nn/quantized/modules/conv.py#L223

Items that needs followup:
* there is a workaround that removes overload (.Tensor) for q/dq ops for the match pattern graph that we get from torchdynamo export, we can remove it after we change the quantized model representation

Test Plan: buck2 test @//mode/opt //caffe2/test:quantization_pt2e -- --exact 'caffe2/test:quantization_pt2e - test_convert_qat_conv_bn_fusion (quantization.pt2e.test_quantize_pt2e.TestQuantizePT2E)'

Reviewed By: kimishpatel

Differential Revision: D45344281

fbshipit-source-id: e5f1edc45ae9606695040469596bb0365f53c4d0

jerryzh168 force-pushed the export-D45344281 branch from f162b5a to f183868 Compare

May 8, 2023 23:53


          [quant][pt2e] Support conv bn fusion in convert step for QAT flow (py…

25f6957

…torch#100442)

Summary:
Pull Request resolved: pytorch#100442

This PR adds support for folding bn weights into conv for QAT flow, this is equivalent
to the QAT branch of `from_float` in eager mode quantized conv module: https://github.com/pytorch/pytorch/blob/main/torch/ao/nn/quantized/modules/conv.py#L223

Items that needs followup:
* there is a workaround that removes overload (.Tensor) for q/dq ops for the match pattern graph that we get from torchdynamo export, we can remove it after we change the quantized model representation

Test Plan: buck2 test @//mode/opt //caffe2/test:quantization_pt2e -- --exact 'caffe2/test:quantization_pt2e - test_convert_qat_conv_bn_fusion (quantization.pt2e.test_quantize_pt2e.TestQuantizePT2E)'

Reviewed By: kimishpatel

Differential Revision: D45344281

fbshipit-source-id: 40c9600220a811a140c3bf1c23851aa3d00766de

Contributor

facebook-github-bot commented May 9, 2023

This pull request was exported from Phabricator. Differential Revision: D45344281

jerryzh168 force-pushed the export-D45344281 branch from f183868 to 25f6957 Compare

May 9, 2023 00:38

Contributor

facebook-github-bot commented May 9, 2023

@pytorchbot merge

(Initiating merge automatically since Phabricator Diff has merged)

pytorch-bot bot added the ciflow/trunk label

pytorchmergebot added the merging label

Collaborator

pytorchmergebot commented May 9, 2023

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

pytorchmergebot closed this in

c3f3cb5

Contributor

facebook-github-bot commented May 9, 2023

This pull request has been merged in c3f3cb5.

facebook-github-bot added the Merged label

pytorchmergebot removed the merging label

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment