Skip to content

[quant][graphmode] Test JIT tracing for dynamic quant cases #40040

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 5 commits into from
Closed

[quant][graphmode] Test JIT tracing for dynamic quant cases #40040

wants to merge 5 commits into from

Conversation

supriyar
Copy link
Contributor

@supriyar supriyar commented Jun 15, 2020 •
edited
Loading

Stack from ghstack:

Summary:

Test Plan:
python test/test_quantization.py TestQuantizeDynamicScriptJitPasses

Reviewers:

Subscribers:

Tasks:

Tags:

Differential Revision: D22071277

@supriyar
[quant][graphmode] Test JIT tracing for dynamic quant cases
eabbe6a 
Summary:

Test Plan:
python test/test_quantization.py TestQuantizeDynamicScriptJitPasses

Reviewers:

Subscribers:

Tasks:

Tags:

[ghstack-poisoned]
@supriyar supriyar mentioned this pull request Jun 15, 2020
Closed
@supriyar supriyar requested a review from jerryzh168 June 15, 2020 18:41
jerryzh168
jerryzh168 reviewed Jun 15, 2020
View reviewed changes
test/quantization/test_quantize_script.py Outdated
Comment on lines 2466 to 2470
if tracing_mode:
x = torch.randn(5, 5)
model = torch.jit.trace(eager_model, x)
else:
model = torch.jit.script(eager_model)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we have a utility function for this: get_script_module. please take a look at other tests in the file.

jerryzh168
jerryzh168 reviewed Jun 15, 2020
View reviewed changes
test/quantization/test_quantize_script.py Outdated
.check("quantized::linear_dynamic") \
.run(model.graph)
eager_model = M().eval()
for tracing_mode in [True, False]:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: suggest to use tracing for the name to align with other tests in the file

@supriyar
Update on "[quant][graphmode] Test JIT tracing for dynamic quant cases"
25d805d 
Summary:

Test Plan:
python test/test_quantization.py TestQuantizeDynamicScriptJitPasses

Reviewers:

Subscribers:

Tasks:

Tags:

[ghstack-poisoned]
@supriyar supriyar requested a review from jerryzh168 June 15, 2020 18:56
@dr-ci
Copy link

dr-ci bot commented Jun 15, 2020 •
edited
Loading

💊 CI failures summary and remediations

As of commit 873131a (more details on the Dr. CI page):

  • 1/1 failures introduced in this PR

🕵️ 1 new failure recognized by patterns

The following CI failures do not appear to be due to upstream breakages:

See CircleCI build pytorch_macos_10_13_py3_test (1/1)

Step: "Test" (full log | diagnosis details | 🔁 rerun)

Jun 16 11:39:40 RuntimeError: test_quantization failed! 
Jun 16 11:39:40 Generated XML report: test-reports/python-unittest/TEST-quantization.test_quantize_script.TestQuantizeScriptPTDQOps-20200616113455.xml 
Jun 16 11:39:40 Generated XML report: test-reports/python-unittest/TEST-quantization.test_quantized_module.TestDynamicQuantizedModule-20200616113455.xml 
Jun 16 11:39:40 Generated XML report: test-reports/python-unittest/TEST-quantization.test_quantize.TestFusion-20200616113455.xml 
Jun 16 11:39:40 Generated XML report: test-reports/python-unittest/TEST-quantization.test_quantize.TestPostTrainingDynamic-20200616113455.xml 
Jun 16 11:39:40 Generated XML report: test-reports/python-unittest/TEST-quantization.test_quantized_op.TestQNNPackOps-20200616113455.xml 
Jun 16 11:39:40 Traceback (most recent call last): 
Jun 16 11:39:40   File "test/run_test.py", line 720, in <module> 
Jun 16 11:39:40     main() 
Jun 16 11:39:40   File "test/run_test.py", line 713, in main 
Jun 16 11:39:40     raise RuntimeError(message) 
Jun 16 11:39:40 RuntimeError: test_quantization failed! 
Jun 16 11:39:40 + cleanup 
Jun 16 11:39:40 + retcode=1 
Jun 16 11:39:40 + set +x
This comment was automatically generated by Dr. CI (expand for details).Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions on the GitHub issue tracker or post in the (internal) Dr. CI Users group.

See how this bot performed.

This comment has been revised 9 times.

jerryzh168
jerryzh168 reviewed Jun 15, 2020
View reviewed changes
test/quantization/test_quantize_script.py Outdated
Comment on lines 2468 to 2471
qconfig_dict = {'' : default_dynamic_qconfig}
model = quantize_dynamic_script(model, qconfig_dict)
# add op is not dynamically quantized.
FileCheck().check("aten::add") \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you use _test_op_impl? It is in TestQuantizeScriptPTSQOps, but you can move it to QuantizationTestCase

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure. That currently only tests quantize_script API. I'll make it generic.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

or maybe define a _test_op_impl locally for this test

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jerryzh168 refactored it a bit to make it usable by both.

@supriyar
Update on "[quant][graphmode] Test JIT tracing for dynamic quant cases"
a9a80cd 
Summary:

Test Plan:
python test/test_quantization.py TestQuantizeDynamicScriptJitPasses

Reviewers:

Subscribers:

Tasks:

Tags:

[ghstack-poisoned]
supriyar added a commit that referenced this pull request Jun 15, 2020
@supriyar
[quant][graphmode] Test JIT tracing for dynamic quant cases
7ad31e9 
Summary:

Test Plan:
python test/test_quantization.py TestQuantizeDynamicScriptJitPasses

Reviewers:

Subscribers:

Tasks:

Tags:

ghstack-source-id: eb24d61
Pull Request resolved: #40040
@supriyar supriyar requested a review from jerryzh168 June 15, 2020 23:37
jerryzh168
jerryzh168 reviewed Jun 16, 2020
View reviewed changes
test/quantization/test_quantize_script.py Outdated
FileCheck().check("aten::add") \
.check("quantized::linear_dynamic") \
.run(model.graph)
eager_model = M().eval()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

btw, eval_mode is an argument of checkGraphModeOp

jerryzh168
jerryzh168 approved these changes Jun 16, 2020
View reviewed changes
Copy link
Contributor

@jerryzh168 jerryzh168 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LG, thanks!

@supriyar
Update on "[quant][graphmode] Test JIT tracing for dynamic quant cases"
1c20d77 
Summary:

Test Plan:
python test/test_quantization.py TestQuantizeDynamicScriptJitPasses

Reviewers:

Subscribers:

Tasks:

Tags:

[ghstack-poisoned]
@supriyar
Update on "[quant][graphmode] Test JIT tracing for dynamic quant cases"
873131a 
Summary:

Test Plan:
python test/test_quantization.py TestQuantizeDynamicScriptJitPasses

Reviewers:

Subscribers:

Tasks:

Tags:

Differential Revision: [D22071277](https://our.internmc.facebook.com/intern/diff/D22071277)

[ghstack-poisoned]
supriyar added a commit that referenced this pull request Jun 16, 2020
@supriyar
[quant][graphmode] Test JIT tracing for dynamic quant cases
d9dd33e 
Summary:

Test Plan:
python test/test_quantization.py TestQuantizeDynamicScriptJitPasses

Reviewers:

Subscribers:

Tasks:

Tags:

ghstack-source-id: 6290497
Pull Request resolved: #40040
@facebook-github-bot
Copy link
Contributor

This pull request has been merged in fa4244d.

xwang233 pushed a commit to xwang233/pytorch that referenced this pull request Jun 20, 2020
@supriyar @xwang233
[quant][graphmode] Test JIT tracing for dynamic quant cases (pytorch#…
1c6cdad 
…40040)

Summary: Pull Request resolved: pytorch#40040

Test Plan:
python test/test_quantization.py TestQuantizeDynamicScriptJitPasses

Imported from OSS

Differential Revision: D22071277

fbshipit-source-id: e8aa8637e6364092b6ff1c3a48dfc4551eb645ec
@facebook-github-bot facebook-github-bot deleted the gh/supriyar/139/head branch June 20, 2020 14:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jerryzh168 jerryzh168 jerryzh168 approved these changes

Assignees

No one assigned

Labels

Merged

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@supriyar @facebook-github-bot @jerryzh168 @mruberry