Enable oneDNN QConv FP32/BF16 output #112010

leslie-fang-intel · 2023-10-25T08:34:01Z

Stack from ghstack (oldest at bottom):

Summary

PR 1 for enabling Int8-Mixed-BF16 PT2E PTQ Quantization with Inductor [RFC] Enable Int8-Mixed-BF16 PT2E PTQ Quantization with Inductor #111640.
Enable QConv (relu, add, add_relu) with BFloat16 or Float32 output.

Test Plan

python -u -m pytest -s -v test_quantized_op.py -k test_qconv1d_pt2e
python -u -m pytest -s -v test_quantized_op.py -k test_qconv2d_pt2e
python -u -m pytest -s -v test_quantized_op.py -k test_qconv3d_pt2e
python -u -m pytest test_quantized_op.py -k test_qconv2d_relu_pt2e
python -u -m pytest test_quantized_op.py -k test_qconv2d_add_pt2e
python -u -m pytest test_quantized_op.py -k test_qconv2d_add_relu_pt2e
python -u -m pytest test_quantized_op.py -k test_qconv2d_add_relu_float_output_pt2e

cc @jgong5 @mingfeima @XiaobingSuper @sanchitintel @ashokei @jingxu10 @voznesenskym @penguinwu @EikanWang @Guobing-Chen @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @yf225 @chenyang78 @kadeng @muchulee8 @aakhundov @ColinPeppler

[ghstack-poisoned]

cc jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 voznesenskym penguinwu EikanWang Guobing-Chen zhuhaozhe blzheng wenzhe-nrv jiayisunx peterbell10 ipiszy yf225 chenyang78 kadeng muchulee8 aakhundov ColinPeppler [ghstack-poisoned]

pytorch-bot · 2023-10-25T08:34:07Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/112010

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (1 Unrelated Failure)

As of commit c13ee83 with merge base 0d95378 ():

FLAKY - The following job failed but was likely due to flakiness present on trunk:

trunk / macos-12-py3-arm64-mps / test (mps, 1, 1, macos-m1-12) (gh)

This comment was automatically generated by Dr. CI and updates every 15 minutes.

cc jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 voznesenskym penguinwu EikanWang Guobing-Chen zhuhaozhe blzheng wenzhe-nrv jiayisunx peterbell10 ipiszy yf225 chenyang78 kadeng muchulee8 aakhundov ColinPeppler [ghstack-poisoned]

**Summary** Enable QConv (relu, add, add_relu) with BFloat16 or Float32 output. **Test Plan** ``` python -u -m pytest -s -v test_quantized_op.py -k test_qconv1d_pt2e python -u -m pytest -s -v test_quantized_op.py -k test_qconv2d_pt2e python -u -m pytest -s -v test_quantized_op.py -k test_qconv3d_pt2e python -u -m pytest test_quantized_op.py -k test_qconv2d_relu_pt2e python -u -m pytest test_quantized_op.py -k test_qconv2d_add_pt2e python -u -m pytest test_quantized_op.py -k test_qconv2d_add_relu_pt2e python -u -m pytest test_quantized_op.py -k test_qconv2d_add_relu_float_output_pt2e ``` cc jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 voznesenskym penguinwu EikanWang Guobing-Chen zhuhaozhe blzheng wenzhe-nrv jiayisunx peterbell10 ipiszy yf225 chenyang78 kadeng muchulee8 aakhundov ColinPeppler [ghstack-poisoned]

ghstack-source-id: 922de9b Pull Request resolved: #112010

aten/src/ATen/native/quantized/cpu/qconv.cpp

**Summary** - PR 1 for enabling Int8-Mixed-BF16 PT2E PTQ Quantization with Inductor #111640. - Enable QConv (relu, add, add_relu) with BFloat16 or Float32 output. **Test Plan** ``` python -u -m pytest -s -v test_quantized_op.py -k test_qconv1d_pt2e python -u -m pytest -s -v test_quantized_op.py -k test_qconv2d_pt2e python -u -m pytest -s -v test_quantized_op.py -k test_qconv3d_pt2e python -u -m pytest test_quantized_op.py -k test_qconv2d_relu_pt2e python -u -m pytest test_quantized_op.py -k test_qconv2d_add_pt2e python -u -m pytest test_quantized_op.py -k test_qconv2d_add_relu_pt2e python -u -m pytest test_quantized_op.py -k test_qconv2d_add_relu_float_output_pt2e ``` cc jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 voznesenskym penguinwu EikanWang Guobing-Chen zhuhaozhe blzheng wenzhe-nrv jiayisunx peterbell10 ipiszy yf225 chenyang78 kadeng muchulee8 aakhundov ColinPeppler [ghstack-poisoned]

ghstack-source-id: af28fba Pull Request resolved: pytorch#112010

**Summary** - PR 1 for enabling Int8-Mixed-BF16 PT2E PTQ Quantization with Inductor #111640. - Enable QConv (relu, add, add_relu) with BFloat16 or Float32 output. **Test Plan** ``` python -u -m pytest -s -v test_quantized_op.py -k test_qconv1d_pt2e python -u -m pytest -s -v test_quantized_op.py -k test_qconv2d_pt2e python -u -m pytest -s -v test_quantized_op.py -k test_qconv3d_pt2e python -u -m pytest test_quantized_op.py -k test_qconv2d_relu_pt2e python -u -m pytest test_quantized_op.py -k test_qconv2d_add_pt2e python -u -m pytest test_quantized_op.py -k test_qconv2d_add_relu_pt2e python -u -m pytest test_quantized_op.py -k test_qconv2d_add_relu_float_output_pt2e ``` cc jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 voznesenskym penguinwu EikanWang Guobing-Chen zhuhaozhe blzheng wenzhe-nrv jiayisunx peterbell10 ipiszy yf225 chenyang78 kadeng muchulee8 aakhundov ColinPeppler [ghstack-poisoned]

ghstack-source-id: 452927e Pull Request resolved: pytorch#112010

leslie-fang-intel · 2023-11-03T08:14:18Z

@pytorchbot merge

pytorchmergebot · 2023-11-03T08:16:12Z

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

**Summary** - PR 2 for enabling Int8-Mixed-BF16 PT2E PTQ Quantization with Inductor #111640. - Enable QLinear (relu) with BFloat16 or Float32 output. **TestPlan** ``` python -u -m pytest -s -v test_quantized_op.py -k test_qlinear_pt2e ``` Pull Request resolved: #112126 Approved by: https://github.com/jerryzh168, https://github.com/jgong5 ghstack dependencies: #112010

…torQuantizer (#112140) **Summary** - PR 3 for enabling Int8-Mixed-BF16 PT2E PTQ Quantization with Inductor #111640. - Remove the output annotation of QConv/QLinear in X86InductorQuantizer. **Test Plan** ``` python -m pytest test_mkldnn_pattern_matcher.py -k test_qconv2d python -m pytest test_mkldnn_pattern_matcher.py -k test_qlinear python -m pytest test_x86inductor_quantizer.py -k Conv2d python -m pytest test_x86inductor_quantizer.py -k Linear ``` Pull Request resolved: #112140 Approved by: https://github.com/jgong5, https://github.com/jerryzh168 ghstack dependencies: #112010, #112126

**Summary** - PR 1 for enabling Int8-Mixed-BF16 PT2E PTQ Quantization with Inductor pytorch#111640. - Enable QConv (relu, add, add_relu) with BFloat16 or Float32 output. **Test Plan** ``` python -u -m pytest -s -v test_quantized_op.py -k test_qconv1d_pt2e python -u -m pytest -s -v test_quantized_op.py -k test_qconv2d_pt2e python -u -m pytest -s -v test_quantized_op.py -k test_qconv3d_pt2e python -u -m pytest test_quantized_op.py -k test_qconv2d_relu_pt2e python -u -m pytest test_quantized_op.py -k test_qconv2d_add_pt2e python -u -m pytest test_quantized_op.py -k test_qconv2d_add_relu_pt2e python -u -m pytest test_quantized_op.py -k test_qconv2d_add_relu_float_output_pt2e ``` Pull Request resolved: pytorch#112010 Approved by: https://github.com/jerryzh168, https://github.com/jgong5

**Summary** - PR 2 for enabling Int8-Mixed-BF16 PT2E PTQ Quantization with Inductor pytorch#111640. - Enable QLinear (relu) with BFloat16 or Float32 output. **TestPlan** ``` python -u -m pytest -s -v test_quantized_op.py -k test_qlinear_pt2e ``` Pull Request resolved: pytorch#112126 Approved by: https://github.com/jerryzh168, https://github.com/jgong5 ghstack dependencies: pytorch#112010

…torQuantizer (pytorch#112140) **Summary** - PR 3 for enabling Int8-Mixed-BF16 PT2E PTQ Quantization with Inductor pytorch#111640. - Remove the output annotation of QConv/QLinear in X86InductorQuantizer. **Test Plan** ``` python -m pytest test_mkldnn_pattern_matcher.py -k test_qconv2d python -m pytest test_mkldnn_pattern_matcher.py -k test_qlinear python -m pytest test_x86inductor_quantizer.py -k Conv2d python -m pytest test_x86inductor_quantizer.py -k Linear ``` Pull Request resolved: pytorch#112140 Approved by: https://github.com/jgong5, https://github.com/jerryzh168 ghstack dependencies: pytorch#112010, pytorch#112126

**Summary** - PR 1 for enabling Int8-Mixed-BF16 PT2E PTQ Quantization with Inductor pytorch#111640. - Enable QConv (relu, add, add_relu) with BFloat16 or Float32 output. **Test Plan** ``` python -u -m pytest -s -v test_quantized_op.py -k test_qconv1d_pt2e python -u -m pytest -s -v test_quantized_op.py -k test_qconv2d_pt2e python -u -m pytest -s -v test_quantized_op.py -k test_qconv3d_pt2e python -u -m pytest test_quantized_op.py -k test_qconv2d_relu_pt2e python -u -m pytest test_quantized_op.py -k test_qconv2d_add_pt2e python -u -m pytest test_quantized_op.py -k test_qconv2d_add_relu_pt2e python -u -m pytest test_quantized_op.py -k test_qconv2d_add_relu_float_output_pt2e ``` Pull Request resolved: pytorch#112010 Approved by: https://github.com/jerryzh168, https://github.com/jgong5

**Summary** - PR 2 for enabling Int8-Mixed-BF16 PT2E PTQ Quantization with Inductor pytorch#111640. - Enable QLinear (relu) with BFloat16 or Float32 output. **TestPlan** ``` python -u -m pytest -s -v test_quantized_op.py -k test_qlinear_pt2e ``` Pull Request resolved: pytorch#112126 Approved by: https://github.com/jerryzh168, https://github.com/jgong5 ghstack dependencies: pytorch#112010

…torQuantizer (pytorch#112140) **Summary** - PR 3 for enabling Int8-Mixed-BF16 PT2E PTQ Quantization with Inductor pytorch#111640. - Remove the output annotation of QConv/QLinear in X86InductorQuantizer. **Test Plan** ``` python -m pytest test_mkldnn_pattern_matcher.py -k test_qconv2d python -m pytest test_mkldnn_pattern_matcher.py -k test_qlinear python -m pytest test_x86inductor_quantizer.py -k Conv2d python -m pytest test_x86inductor_quantizer.py -k Linear ``` Pull Request resolved: pytorch#112140 Approved by: https://github.com/jgong5, https://github.com/jerryzh168 ghstack dependencies: pytorch#112010, pytorch#112126

leslie-fang-intel added 2 commits October 25, 2023 16:30

Enable onednn.QConv FP32/BF16 output

71fa799

[ghstack-poisoned]

leslie-fang-intel requested review from jerryzh168, salilsdesai, kimishpatel, digantdesai and jianyuh as code owners October 25, 2023 08:34

pytorch-bot bot added the release notes: quantization release notes category label Oct 25, 2023

github-actions bot added module: cpu CPU specific problem (e.g., perf, algorithm) module: inductor ciflow/inductor labels Oct 25, 2023

leslie-fang-intel marked this pull request as draft October 25, 2023 08:34

leslie-fang-intel changed the title ~~Enable onednn.QConv FP32/BF16 output~~ [WIP] Enable onednn.QConv FP32/BF16 output Oct 25, 2023

leslie-fang-intel added the ciflow/trunk Trigger trunk jobs on your pull request label Oct 25, 2023

leslie-fang-intel changed the title ~~[WIP] Enable onednn.QConv FP32/BF16 output~~ [WIP] Enable onednn QConv FP32/BF16 output Oct 25, 2023

pytorchbot added the open source label Oct 25, 2023

leslie-fang-intel changed the title ~~[WIP] Enable onednn QConv FP32/BF16 output~~ Enable onednn QConv FP32/BF16 output Oct 26, 2023

leslie-fang-intel added a commit that referenced this pull request Oct 26, 2023

Enable onednn.QConv FP32/BF16 output

e74aa71

ghstack-source-id: 922de9b Pull Request resolved: #112010

This was referenced Oct 26, 2023

Enable oneDNN QLinear FP32/BF16 output #112126

Closed

[Quant] [PT2] Remove the output Annotation of Conv/Linear in x86InductorQuantizer #112140

Closed

[Quant] [PT2] Enable Decomposed quant per tensor/channel to accept bfloat16 input #112225

Closed

jerryzh168 approved these changes Oct 31, 2023

View reviewed changes

leslie-fang-intel changed the title ~~Enable onednn QConv FP32/BF16 output~~ Enable oneDNN QConv FP32/BF16 output Oct 31, 2023

This was referenced Oct 31, 2023

[Inductor] [Quant] Enable QConv2d int8-mixed-bf16 Lowering #112469

Closed

[Inductor] [Quant] Enable QLinear int8-mixed-bf16 Lowering #112486

Closed

leslie-fang-intel requested review from jgong5 and Xia-Weiwen November 1, 2023 02:13

This was referenced Nov 1, 2023

[Inductor] [Quant] Enable QConv2d Unary int8-mixed-bf16 Lowering #112550

Closed

[Inductor] [Quant] Enable QConv2d Binary int8-mixed-bf16 Lowering #112551

Closed

leslie-fang-intel marked this pull request as ready for review November 1, 2023 02:14

jgong5 approved these changes Nov 1, 2023

View reviewed changes

aten/src/ATen/native/quantized/cpu/qconv.cpp Outdated Show resolved Hide resolved

leslie-fang-intel added a commit to leslie-fang-intel/pytorch that referenced this pull request Nov 3, 2023

Enable onednn.QConv FP32/BF16 output

efacc46

ghstack-source-id: af28fba Pull Request resolved: pytorch#112010

leslie-fang-intel added a commit to leslie-fang-intel/pytorch that referenced this pull request Nov 3, 2023

Enable onednn.QConv FP32/BF16 output

6a3b887

ghstack-source-id: 452927e Pull Request resolved: pytorch#112010

pytorchmergebot added the merging label Nov 3, 2023

pytorchmergebot added the Merged label Nov 3, 2023

pytorchmergebot closed this in b6fc7af Nov 3, 2023

pytorchmergebot removed the merging label Nov 3, 2023

facebook-github-bot deleted the gh/leslie-fang-intel/33/head branch November 6, 2023 15:25

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Enable oneDNN QConv FP32/BF16 output #112010

Enable oneDNN QConv FP32/BF16 output #112010

Uh oh!

leslie-fang-intel commented Oct 25, 2023 •

edited

Loading

Uh oh!

pytorch-bot bot commented Oct 25, 2023 •

edited

Loading

Uh oh!

Uh oh!

leslie-fang-intel commented Nov 3, 2023

Uh oh!

pytorchmergebot commented Nov 3, 2023

Uh oh!

Uh oh!

Enable oneDNN QConv FP32/BF16 output #112010

Enable oneDNN QConv FP32/BF16 output #112010

Uh oh!

Conversation

leslie-fang-intel commented Oct 25, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot bot commented Oct 25, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/112010

✅ You can merge normally! (1 Unrelated Failure)

Uh oh!

Uh oh!

leslie-fang-intel commented Nov 3, 2023

Uh oh!

pytorchmergebot commented Nov 3, 2023

Merge started

Uh oh!

Uh oh!

leslie-fang-intel commented Oct 25, 2023 •

edited

Loading

pytorch-bot bot commented Oct 25, 2023 •

edited

Loading