[Quant][FX] Lower QConvAdd2d for onednn backend #91153

leslie-fang-intel · 2022-12-20T03:55:03Z

Stack from ghstack (oldest at bottom):

Summary
Add quantization mappings for QConvAdd2d for int8 inference for onednn backend. The fusion and lowering is supported only in FX mode.

Test plan

python -m pytest test_quantization.py -k test_fuse_conv_bn_add_relu_onednn
python -m pytest test_quantization.py -k test_fuse_conv_bn_add_relu_by_default
python -m pytest test_quantization.py -k test_fuse_conv_bn_add_relu_lowering

cc @jgong5 @mingfeima @XiaobingSuper @sanchitintel @ashokei @jingxu10

[ghstack-poisoned]

pytorch-bot · 2022-12-20T03:55:06Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/91153

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 2cf5520:
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

ghstack-source-id: 0d61a8fefb95aa4f1655440fb5b59fb576b3935f Pull Request resolved: pytorch#91153

cc jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 [ghstack-poisoned]

ghstack-source-id: e47b196a6f8a528c42bfbb90665301cbb7d093b6 Pull Request resolved: pytorch#91153

cc jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 [ghstack-poisoned]

ghstack-source-id: af68a2315c063284149a5ed21da3e526746909ac Pull Request resolved: pytorch#91153

cc jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 [ghstack-poisoned]

**Summary** Add quantization mappings for QConvAdd2d for int8 inference for onednn backend. The fusion and lowering is supported only in FX mode. **Test plan** ``` python -m pytest test_quantization.py -k test_fuse_conv_bn_add_relu_onednn python -m pytest test_quantization.py -k test_fuse_conv_bn_add_relu_by_default python -m pytest test_quantization.py -k test_fuse_conv_bn_add_relu_lowering ``` cc jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 [ghstack-poisoned]

leslie-fang-intel · 2023-01-11T06:12:54Z

@jerryzh168 I have changed according to your comments. Could you help to take a look of this PR again?

ghstack-source-id: fb7521b7c1e80757744e17fc85888f23c9987601 Pull Request resolved: pytorch#91153

**Summary** Add quantization mappings for QConvAdd2d for int8 inference for onednn backend. The fusion and lowering is supported only in FX mode. **Test plan** ``` python -m pytest test_quantization.py -k test_fuse_conv_bn_add_relu_onednn python -m pytest test_quantization.py -k test_fuse_conv_bn_add_relu_by_default python -m pytest test_quantization.py -k test_fuse_conv_bn_add_relu_lowering ``` cc jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 [ghstack-poisoned]

leslie-fang-intel · 2023-01-26T02:23:41Z

Hi @jerryzh168 Is there any other comments to this PR? Could you help to take a look again?

**Summary** Add quantization mappings for QConvAdd2d for int8 inference for onednn backend. The fusion and lowering is supported only in FX mode. **Test plan** ``` python -m pytest test_quantization.py -k test_fuse_conv_bn_add_relu_onednn python -m pytest test_quantization.py -k test_fuse_conv_bn_add_relu_by_default python -m pytest test_quantization.py -k test_fuse_conv_bn_add_relu_lowering ``` cc jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 [ghstack-poisoned]

leslie-fang-intel · 2023-01-31T01:21:06Z

Hi @jerryzh168, Could you also take a review of this PR again?

jerryzh168 · 2023-01-31T01:44:16Z

test/quantization/core/test_quantized_module.py

-            # workaround in this PR to return from here, since the below lowering part enabled in next PR
-            # We will enable below check in next PR
-            return
+        class _FusedModule_two_input_args(torch.nn.intrinsic._FusedModule):


do we need this?

Yes, I think so. That's because original torch.nn.intrinsic._FusedModule only support one input.

jerryzh168 · 2023-01-31T01:45:07Z

test/quantization/fx/test_quantize_fx.py

+
+        options = itertools.product(
+            [True, False],  # with_bn
+            [False],  # with_relu


is relu supported? the name of the test mentioned relu, but we did not test it here

Oh, yes. The support of relu is added in the following up 2 PRs.

jerryzh168 · 2023-01-31T01:45:41Z

test/quantization/fx/test_quantize_fx.py

+    def test_fuse_conv_bn_add_relu_by_default(self):
+        options = itertools.product(
+            [True, False],  # with_bn
+            [False],  # with_relu


same question for relu

The support of relu has been added in the following up PRs.

jerryzh168 · 2023-01-31T01:47:47Z

torch/ao/quantization/fx/_lower_to_native_backend.py

@@ -268,6 +268,15 @@ def should_skip_lowering(op: torch.fx.node.Node, qconfig_map: Dict[str, QConfigA
    nni.ConvReLU3d: (nnqr.Conv3d, nniq.ConvReLU3d),
 }

+# The difference between STATIC_LOWER_FUSED_MODULE_TWO_INPUTS_MAP and STATIC_LOWER_FUSED_MODULE_MAP:
+# The refer node inside STATIC_LOWER_FUSED_MODULE_TWO_INPUTS_MAP has 2 dq input nodes.


nit: dq input nodes --> inputs

Thanks for the comments and changed.

jerryzh168

lgtm, thanks!

**Summary** Add quantization mappings for QConvAdd2d for int8 inference for onednn backend. The fusion and lowering is supported only in FX mode. **Test plan** ``` python -m pytest test_quantization.py -k test_fuse_conv_bn_add_relu_onednn python -m pytest test_quantization.py -k test_fuse_conv_bn_add_relu_by_default python -m pytest test_quantization.py -k test_fuse_conv_bn_add_relu_lowering ``` cc jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 [ghstack-poisoned]

leslie-fang-intel · 2023-02-01T01:11:52Z

@pytorchbot merge

pytorchmergebot · 2023-02-01T01:14:03Z

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

[Quant][FX] Lower QConvAdd2d for onednn backend

5eb0224

[ghstack-poisoned]

pytorch-bot bot added the release notes: AO frontend label Dec 20, 2022

github-actions bot added the release notes: quantization release notes category label Dec 20, 2022

leslie-fang-intel requested review from jgong5 and Xia-Weiwen December 20, 2022 03:57

leslie-fang-intel marked this pull request as draft December 20, 2022 03:57

leslie-fang-intel changed the title ~~[Quant][FX] Lower QConvAdd2d for onednn backend~~ [WIP][Quant][FX] Lower QConvAdd2d for onednn backend Dec 20, 2022

leslie-fang-intel added ciflow/trunk Trigger trunk jobs on your pull request intel This tag is for PR from Intel labels Dec 20, 2022

pytorchbot added the open source label Dec 20, 2022

leslie-fang-intel added a commit to leslie-fang-intel/pytorch that referenced this pull request Dec 20, 2022

[Quant][FX] Lower QConvAdd2d for onednn backend

e3572eb

ghstack-source-id: 0d61a8fefb95aa4f1655440fb5b59fb576b3935f Pull Request resolved: pytorch#91153

Update on "[WIP][Quant][FX] Lower QConvAdd2d for onednn backend"

b0caac8

cc jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 [ghstack-poisoned]

leslie-fang-intel added a commit to leslie-fang-intel/pytorch that referenced this pull request Dec 20, 2022

[Quant][FX] Lower QConvAdd2d for onednn backend

d4e28f2

ghstack-source-id: e47b196a6f8a528c42bfbb90665301cbb7d093b6 Pull Request resolved: pytorch#91153

leslie-fang-intel added 2 commits December 20, 2022 15:11

Update on "[WIP][Quant][FX] Lower QConvAdd2d for onednn backend"

05e9705

cc jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 [ghstack-poisoned]

Update on "[WIP][Quant][FX] Lower QConvAdd2d for onednn backend"

6308ff5

cc jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 [ghstack-poisoned]

leslie-fang-intel added a commit to leslie-fang-intel/pytorch that referenced this pull request Dec 20, 2022

[Quant][FX] Lower QConvAdd2d for onednn backend

2b039dd

ghstack-source-id: af68a2315c063284149a5ed21da3e526746909ac Pull Request resolved: pytorch#91153

leslie-fang-intel added 5 commits December 20, 2022 16:44

Update on "[WIP][Quant][FX] Lower QConvAdd2d for onednn backend"

bddb6e2

cc jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 [ghstack-poisoned]

Update on "[WIP][Quant][FX] Lower QConvAdd2d for onednn backend"

0682977

cc jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 [ghstack-poisoned]

Update on "[WIP][Quant][FX] Lower QConvAdd2d for onednn backend"

9cb9b11

cc jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 [ghstack-poisoned]

Update on "[WIP][Quant][FX] Lower QConvAdd2d for onednn backend"

169dedb

cc jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 [ghstack-poisoned]

Update on "[WIP][Quant][FX] Lower QConvAdd2d for onednn backend"

2f21bac

cc jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 [ghstack-poisoned]

leslie-fang-intel requested review from jerryzh168 and removed request for z-a-f January 11, 2023 01:45

leslie-fang-intel added a commit to leslie-fang-intel/pytorch that referenced this pull request Jan 13, 2023

[Quant][FX] Lower QConvAdd2d for onednn backend

dbf5033

ghstack-source-id: fb7521b7c1e80757744e17fc85888f23c9987601 Pull Request resolved: pytorch#91153

leslie-fang-intel added a commit to leslie-fang-intel/pytorch that referenced this pull request Jan 26, 2023

[Quant][FX] Lower QConvAdd2d for onednn backend

461dbc2

ghstack-source-id: fb7521b7c1e80757744e17fc85888f23c9987601 Pull Request resolved: pytorch#91153

leslie-fang-intel added 8 commits January 26, 2023 10:35

jerryzh168 reviewed Jan 31, 2023

View reviewed changes

jerryzh168 approved these changes Jan 31, 2023

View reviewed changes

leslie-fang-intel added 2 commits January 31, 2023 11:03

pytorchmergebot added the Merged label Feb 1, 2023

pytorchmergebot closed this in ef4118e Feb 1, 2023

facebook-github-bot deleted the gh/leslie-fang-intel/15/head branch June 8, 2023 17:52

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Quant][FX] Lower QConvAdd2d for onednn backend #91153

[Quant][FX] Lower QConvAdd2d for onednn backend #91153

leslie-fang-intel commented Dec 20, 2022 •

edited

pytorch-bot bot commented Dec 20, 2022 •

edited

leslie-fang-intel commented Jan 11, 2023

leslie-fang-intel commented Jan 26, 2023

leslie-fang-intel commented Jan 31, 2023

jerryzh168 Jan 31, 2023

leslie-fang-intel Jan 31, 2023

jerryzh168 Jan 31, 2023 •

edited

leslie-fang-intel Jan 31, 2023

jerryzh168 Jan 31, 2023

leslie-fang-intel Jan 31, 2023

jerryzh168 Jan 31, 2023 •

edited

leslie-fang-intel Jan 31, 2023

jerryzh168 left a comment

leslie-fang-intel commented Feb 1, 2023

pytorchmergebot commented Feb 1, 2023

[Quant][FX] Lower QConvAdd2d for onednn backend #91153

[Quant][FX] Lower QConvAdd2d for onednn backend #91153

Conversation

leslie-fang-intel commented Dec 20, 2022 • edited

pytorch-bot bot commented Dec 20, 2022 • edited

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/91153

✅ No Failures

leslie-fang-intel commented Jan 11, 2023

leslie-fang-intel commented Jan 26, 2023

leslie-fang-intel commented Jan 31, 2023

jerryzh168 Jan 31, 2023

Choose a reason for hiding this comment

leslie-fang-intel Jan 31, 2023

Choose a reason for hiding this comment

jerryzh168 Jan 31, 2023 • edited

Choose a reason for hiding this comment

leslie-fang-intel Jan 31, 2023

Choose a reason for hiding this comment

jerryzh168 Jan 31, 2023

Choose a reason for hiding this comment

leslie-fang-intel Jan 31, 2023

Choose a reason for hiding this comment

jerryzh168 Jan 31, 2023 • edited

Choose a reason for hiding this comment

leslie-fang-intel Jan 31, 2023

Choose a reason for hiding this comment

jerryzh168 left a comment

Choose a reason for hiding this comment

leslie-fang-intel commented Feb 1, 2023

pytorchmergebot commented Feb 1, 2023

Merge started

leslie-fang-intel commented Dec 20, 2022 •

edited

pytorch-bot bot commented Dec 20, 2022 •

edited

jerryzh168 Jan 31, 2023 •

edited

jerryzh168 Jan 31, 2023 •

edited