[quant][pt2e] store scale/zero_point as tensor attributes to support serialization #105894

jerryzh168 · 2023-07-25T03:39:45Z

Stack from ghstack (oldest at bottom):

-> [quant][pt2e] store scale/zero_point as tensor attributes to support serialization #105894

Summary:
Currently scale/zero_point for per tensor quant is stored as burnt in literals, this means these values can't be serialized in state_dict, this
PR changes them to buffers/Tensors so that they can be serialized

Test Plan:
python test/test_quantization.py TestQuantizePT2E

Reviewers:

Subscribers:

Tasks:

Tags:

…serialization Summary: Currently scale/zero_point for per tensor quant is stored as burnt in literals, this means these values can't be serialized in state_dict, this PR changes them to buffers/Tensors so that they can be serialized Test Plan: python test/test_quantization.py TestQuantizePT2E Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned]

pytorch-bot · 2023-07-25T03:39:47Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/105894

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

✅ 2 Unrelated Failures

As of commit ac16b45:

UNSTABLE - The following jobs failed but were likely due to flakiness present on trunk and has been marked as unstable:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

…to support serialization" Summary: Currently scale/zero_point for per tensor quant is stored as burnt in literals, this means these values can't be serialized in state_dict, this PR changes them to buffers/Tensors so that they can be serialized Test Plan: python test/test_quantization.py TestQuantizePT2E Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned]

…serialization Summary: Currently scale/zero_point for per tensor quant is stored as burnt in literals, this means these values can't be serialized in state_dict, this PR changes them to buffers/Tensors so that they can be serialized Test Plan: python test/test_quantization.py TestQuantizePT2E Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 5522977 Pull Request resolved: #105894

jerryzh168 · 2023-07-25T20:00:54Z

@jerryzh168 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

…to support serialization" Summary: Currently scale/zero_point for per tensor quant is stored as burnt in literals, this means these values can't be serialized in state_dict, this PR changes them to buffers/Tensors so that they can be serialized Test Plan: python test/test_quantization.py TestQuantizePT2E Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D47770963](https://our.internmc.facebook.com/intern/diff/D47770963) [ghstack-poisoned]

…serialization Summary: Currently scale/zero_point for per tensor quant is stored as burnt in literals, this means these values can't be serialized in state_dict, this PR changes them to buffers/Tensors so that they can be serialized Test Plan: python test/test_quantization.py TestQuantizePT2E Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: e281fa3 Pull Request resolved: #105894

jerryzh168 · 2023-07-25T20:14:59Z

@jerryzh168 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

jerryzh168 · 2023-07-26T16:20:11Z

@pytorchbot merge

pytorchmergebot · 2023-07-26T16:22:00Z

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

PaliC · 2023-07-27T23:23:57Z

@pytorchbot revert -m "breaking executorch tests internally" -c ghfirst

pytorchmergebot · 2023-07-27T23:25:45Z

@pytorchbot successfully started a revert job. Check the current status here.
Questions? Feedback? Please reach out to the PyTorch DevX Team

pytorchmergebot · 2023-07-27T23:25:51Z

Can't revert PR that was landed via phabricator as D47770963. Please revert by going to the internal diff and clicking Unland.

PaliC · 2023-07-28T00:44:17Z

@pytorchbot revert -m "breaking executorch tests internally" -c ghfirst

pytorchmergebot · 2023-07-28T00:45:51Z

@pytorchbot successfully started a revert job. Check the current status here.
Questions? Feedback? Please reach out to the PyTorch DevX Team

pytorchmergebot · 2023-07-28T00:45:56Z

Can't revert PR that was landed via phabricator as D47770963. Please revert by going to the internal diff and clicking Unland.

pytorchmergebot · 2023-07-28T01:11:15Z

Can't revert PR that was landed via phabricator as D47770963. Please revert by going to the internal diff and clicking Unland.

huydhn · 2023-07-28T01:14:16Z

@pytorchbot revert -m "breaking executorch tests internally" -c ghfirst

pytorchmergebot · 2023-07-28T01:15:56Z

@pytorchbot successfully started a revert job. Check the current status here.
Questions? Feedback? Please reach out to the PyTorch DevX Team

pytorchmergebot · 2023-07-28T01:16:06Z

@jerryzh168 your PR has been successfully reverted.

…support serialization (#105894)" This reverts commit 3ca71ed. Reverted #105894 on behalf of https://github.com/huydhn due to breaking executorch tests internally ([comment](#105894 (comment)))

…support serialization (#105894)" This reverts commit 3ca71ed. [ghstack-poisoned]

…support serialization (#105894)" This reverts commit 3ca71ed. ghstack-source-id: e55f2b4 Pull Request resolved: #106184

leslie-fang-intel · 2023-07-28T06:16:57Z

Hi @jerryzh168, I think this PR has broken quantization inductor flow in this ghstack #105996. Can we have more discussion about the solution before re-landing of this PR? cc @jgong5 @Guobing-Chen

…serialization (pytorch#105894) Summary: Currently scale/zero_point for per tensor quant is stored as burnt in literals, this means these values can't be serialized in state_dict, this PR changes them to buffers/Tensors so that they can be serialized Test Plan: python test/test_quantization.py TestQuantizePT2E Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D47770963](https://our.internmc.facebook.com/intern/diff/D47770963) Pull Request resolved: pytorch#105894 Approved by: https://github.com/kimishpatel

jerryzh168 · 2023-07-28T17:09:38Z

Hi @jerryzh168, I think this PR has broken quantization inductor flow in this ghstack #105996. Can we have more discussion about the solution before re-landing of this PR? cc @jgong5 @Guobing-Chen

sure, this changes quantize_per_tensor.default to quantize_per_tensor.tensor and that's pretty much it, is there anything else that's broken?

…support serialization (pytorch#105894)" This reverts commit 3ca71ed. Reverted pytorch#105894 on behalf of https://github.com/huydhn due to breaking executorch tests internally ([comment](pytorch#105894 (comment)))

**Summary** Draft the fix of QConv lowering in Inductor after PR #105894 cc jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 voznesenskym penguinwu EikanWang Guobing-Chen zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx peterbell10 ipiszy ngimel yf225 chenyang78 kadeng muchulee8 aakhundov [ghstack-poisoned]

…alar tensor" **Summary** Draft the fix of QConv lowering in Inductor after PR #105894 cc jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 voznesenskym penguinwu EikanWang Guobing-Chen zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx peterbell10 ipiszy ngimel yf225 chenyang78 kadeng muchulee8 aakhundov [ghstack-poisoned]

**Summary** Cherry-pick #105894 for further testing. [ghstack-poisoned]

**Summary** Draft the fix of QConv lowering in Inductor after PR #105894 cc jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 voznesenskym penguinwu EikanWang Guobing-Chen zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx peterbell10 ipiszy ngimel yf225 chenyang78 kadeng muchulee8 aakhundov [ghstack-poisoned]

**Summary** Cherry-pick #105894 for further testing. [ghstack-poisoned]

…support serialization (pytorch#105894) Summary: Currently scale/zero_point for per tensor quant is stored as burnt in literals, this means these values can't be serialized in state_dict, this PR changes them to buffers/Tensors so that they can be serialized Test Plan: contbuild & OSS CI, see https://hud.pytorch.org/commit/pytorch/pytorch/3ca71ed735257cb7ad377b57a45057c265893a40 Test plan from GitHub: python test/test_quantization.py TestQuantizePT2E Original Phabricator Test Plan: python test/test_quantization.py TestQuantizePT2E Imported from OSS Differential Revision: D47933210 fbshipit-source-id: 6c0993e58c5f8fd95c4d57b0fdb51e14a3573989

**Summary** Cherry-pick #105894 for further testing. [ghstack-poisoned]

…alar tensor" **Summary** Draft the fix of QConv lowering in Inductor after PR #105894 cc jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 voznesenskym penguinwu EikanWang Guobing-Chen zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx peterbell10 ipiszy ngimel yf225 chenyang78 kadeng muchulee8 aakhundov [ghstack-poisoned]

**Summary** Draft the fix of QConv lowering in Inductor after PR #105894 cc jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 voznesenskym penguinwu EikanWang Guobing-Chen zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx peterbell10 ipiszy ngimel yf225 chenyang78 kadeng muchulee8 aakhundov [ghstack-poisoned]

…alar tensor" **Summary** Draft the fix of QConv lowering in Inductor after PR #105894 cc jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 voznesenskym penguinwu EikanWang Guobing-Chen zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx peterbell10 ipiszy ngimel yf225 chenyang78 kadeng muchulee8 aakhundov [ghstack-poisoned]

**Summary** Draft the fix of QConv lowering in Inductor after PR #105894 cc jgong5 mingfeima XiaobingSuper sanchitintel ashokei jingxu10 voznesenskym penguinwu EikanWang Guobing-Chen zhuhaozhe blzheng Xia-Weiwen wenzhe-nrv jiayisunx peterbell10 ipiszy ngimel yf225 chenyang78 kadeng muchulee8 aakhundov [ghstack-poisoned]

pytorch-bot bot added release notes: quantization release notes category labels Jul 25, 2023

jerryzh168 requested review from andrewor14 and kimishpatel July 25, 2023 03:41

kimishpatel approved these changes Jul 25, 2023

View reviewed changes

pytorch-bot bot added the ciflow/trunk Trigger trunk jobs on your pull request label Jul 26, 2023

pytorchmergebot added the merging label Jul 26, 2023

pytorchmergebot added Merged and removed merging labels Jul 26, 2023

pytorchmergebot closed this in 3ca71ed Jul 26, 2023

pytorchmergebot added the Reverted label Jul 28, 2023

PaliC added a commit that referenced this pull request Jul 28, 2023

Revert "[quant][pt2e] store scale/zero_point as tensor attributes to …

0b290e7

…support serialization (#105894)" This reverts commit 3ca71ed. [ghstack-poisoned]

PaliC added a commit that referenced this pull request Jul 28, 2023

Revert "[quant][pt2e] store scale/zero_point as tensor attributes to …

efa5426

…support serialization (#105894)" This reverts commit 3ca71ed. ghstack-source-id: e55f2b4 Pull Request resolved: #106184

PaliC mentioned this pull request Jul 28, 2023

Revert "[quant][pt2e] store scale/zero_point as tensor attributes to support serialization (#105894)" #106184

Closed

leslie-fang-intel mentioned this pull request Jul 28, 2023

Scalar Tensor lowering to Fake Tensor inside Inductor #106197

Closed

This was referenced Jul 30, 2023

[Test Only] Apply patch 105894 #106263

Closed

[Test Only] Enable lowering of scale as scalar tensor #106264

Closed

leslie-fang-intel added a commit that referenced this pull request Aug 1, 2023

Update base for Update on "[Test Only] Apply patch 105894"

1ba1555

**Summary** Cherry-pick #105894 for further testing. [ghstack-poisoned]

leslie-fang-intel added a commit that referenced this pull request Aug 1, 2023

Update on "[Test Only] Apply patch 105894"

6838552

**Summary** Cherry-pick #105894 for further testing. [ghstack-poisoned]

jerryzh168 closed this Aug 2, 2023

leslie-fang-intel added a commit that referenced this pull request Aug 3, 2023

Update base for Update on "[Test Only] Apply patch 105894"

1a7169c

**Summary** Cherry-pick #105894 for further testing. [ghstack-poisoned]

leslie-fang-intel added a commit that referenced this pull request Aug 3, 2023

Update on "[Test Only] Apply patch 105894"

82ee99f

**Summary** Cherry-pick #105894 for further testing. [ghstack-poisoned]

facebook-github-bot deleted the gh/jerryzh168/894/head branch August 5, 2023 14:17

[quant][pt2e] store scale/zero_point as tensor attributes to support serialization #105894

[quant][pt2e] store scale/zero_point as tensor attributes to support serialization #105894

Uh oh!

Conversation

jerryzh168 commented Jul 25, 2023 • edited by huydhn Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot bot commented Jul 25, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/105894

✅ 2 Unrelated Failures

Uh oh!

jerryzh168 commented Jul 25, 2023

Uh oh!

jerryzh168 commented Jul 25, 2023

Uh oh!

jerryzh168 commented Jul 26, 2023

Uh oh!

pytorchmergebot commented Jul 26, 2023

Merge started

Uh oh!

PaliC commented Jul 27, 2023

Uh oh!

pytorchmergebot commented Jul 27, 2023

Uh oh!

pytorchmergebot commented Jul 27, 2023

Uh oh!

PaliC commented Jul 28, 2023

Uh oh!

pytorchmergebot commented Jul 28, 2023

Uh oh!

pytorchmergebot commented Jul 28, 2023

Uh oh!

pytorchmergebot commented Jul 28, 2023

Uh oh!

huydhn commented Jul 28, 2023

Uh oh!

pytorchmergebot commented Jul 28, 2023

Uh oh!

pytorchmergebot commented Jul 28, 2023

Uh oh!

leslie-fang-intel commented Jul 28, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jerryzh168 commented Jul 28, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

jerryzh168 commented Jul 25, 2023 •

edited by huydhn

Loading

pytorch-bot bot commented Jul 25, 2023 •

edited

Loading

leslie-fang-intel commented Jul 28, 2023 •

edited

Loading