Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[quant][graphmode][fx] Allow user to specify qconfig for call_method #49621

Closed
wants to merge 3 commits into from

Conversation

jerryzh168
Copy link
Contributor

@jerryzh168 jerryzh168 commented Dec 19, 2020

Stack from ghstack:

Summary:
This adds support to configure qconfig for a call_method, e.g. x.chunk, this will help workaround
a problem in our internal model.

TODO: since call_method is also a string and we flatten the qconfig, might need to resolve namespace conflict between
call_method and module_name
TODO: Add scope support to set the qconfig for call_method correctly with original qconfig

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:

Differential Revision: D25651828

Summary:
This adds support to configure qconfig for a call_method, e.g. x.chunk, this will help workaround
a problem in our internal model.

TODO: since call_method is also a string and we flatten the qconfig, might need to resolve namespace conflict between
call_method and module_name
TODO: Add scope support to set the qconfig for call_method correctly with original qconfig

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:

[ghstack-poisoned]
@facebook-github-bot
Copy link
Contributor

facebook-github-bot commented Dec 19, 2020

💊 CI failures summary and remediations

As of commit 8669a27 (more details on the Dr. CI page):


  • 6/7 failures possibly* introduced in this PR
    • 1/6 non-CircleCI failure(s)
  • 1/7 broken upstream at merge base b8d98f0 on Dec 17 from 12:59am to 12:47pm

🕵️ 4 new failures recognized by patterns

The following CI failures do not appear to be due to upstream breakages:

See CircleCI build pytorch_linux_xenial_py3_6_gcc5_4_test (1/4)

Step: "Report results" (full log | diagnosis details | 🔁 rerun)

Dec 19 03:09:19 ModuleNotFoundError: No module named 'boto3'
Dec 19 03:09:19 + CIRCLE_BRANCH=gh/jerryzh168/524/head
Dec 19 03:09:19 + export CIRCLE_JOB=pytorch_linux_xenial_py3_6_gcc5_4_test
Dec 19 03:09:19 + CIRCLE_JOB=pytorch_linux_xenial_py3_6_gcc5_4_test
Dec 19 03:09:19 + export CIRCLE_WORKFLOW_ID=539f9711-13dd-49ef-9742-1e2663af2630
Dec 19 03:09:19 + CIRCLE_WORKFLOW_ID=539f9711-13dd-49ef-9742-1e2663af2630
Dec 19 03:09:19 + cd workspace
Dec 19 03:09:19 + python test/print_test_stats.py test
Dec 19 03:09:19 Traceback (most recent call last):
Dec 19 03:09:19   File "test/print_test_stats.py", line 12, in <module>
Dec 19 03:09:19     import boto3
Dec 19 03:09:19 ModuleNotFoundError: No module named 'boto3'


Exited with code exit status 1

See CircleCI build pytorch_xla_linux_bionic_py3_6_clang9_test (2/4)

Step: "Report results" (full log | diagnosis details | 🔁 rerun)

Dec 19 03:21:12 ModuleNotFoundError: No module named 'boto3'
Dec 19 03:21:12 + CIRCLE_BRANCH=gh/jerryzh168/524/head
Dec 19 03:21:12 + export CIRCLE_JOB=pytorch_xla_linux_bionic_py3_6_clang9_test
Dec 19 03:21:12 + CIRCLE_JOB=pytorch_xla_linux_bionic_py3_6_clang9_test
Dec 19 03:21:12 + export CIRCLE_WORKFLOW_ID=539f9711-13dd-49ef-9742-1e2663af2630
Dec 19 03:21:12 + CIRCLE_WORKFLOW_ID=539f9711-13dd-49ef-9742-1e2663af2630
Dec 19 03:21:12 + cd workspace
Dec 19 03:21:12 + python test/print_test_stats.py test
Dec 19 03:21:12 Traceback (most recent call last):
Dec 19 03:21:12   File "test/print_test_stats.py", line 12, in <module>
Dec 19 03:21:12     import boto3
Dec 19 03:21:12 ModuleNotFoundError: No module named 'boto3'


Exited with code exit status 1

See CircleCI build pytorch_linux_bionic_py3_8_gcc9_coverage_test2 (3/4)

Step: "Run tests" (full log | diagnosis details | 🔁 rerun)

Dec 19 02:29:43 [E request_callback_no_python.cpp:636] Received error while processing request type 258: RuntimeError: Can not pickle torch.futures.Future
Dec 19 02:29:43 At:
Dec 19 02:29:43   /opt/conda/lib/python3.8/site-packages/torch/distributed/rpc/internal.py(120): serialize
Dec 19 02:29:43   /opt/conda/lib/python3.8/site-packages/torch/distributed/rpc/internal.py(172): serialize
Dec 19 02:29:43 
Dec 19 02:29:43 [E request_callback_no_python.cpp:636] Received error while processing request type 258: RuntimeError: Can not pickle torch.futures.Future
Dec 19 02:29:43 
Dec 19 02:29:43 At:
Dec 19 02:29:43   /opt/conda/lib/python3.8/site-packages/torch/distributed/rpc/internal.py(120): serialize
Dec 19 02:29:43   /opt/conda/lib/python3.8/site-packages/torch/distributed/rpc/internal.py(172): serialize
Dec 19 02:29:43 
Dec 19 02:29:43 [E request_callback_no_python.cpp:636] Received error while processing request type 258: RuntimeError: Can not pickle torch.futures.Future
Dec 19 02:29:43 
Dec 19 02:29:43 At:
Dec 19 02:29:43   /opt/conda/lib/python3.8/site-packages/torch/distributed/rpc/internal.py(120): serialize
Dec 19 02:29:43   /opt/conda/lib/python3.8/site-packages/torch/distributed/rpc/internal.py(172): serialize
Dec 19 02:29:43 
Dec 19 02:29:43 [W tensorpipe_agent.cpp:547] RPC agent for worker3 encountered error when reading incoming request from worker2: EOF: end of file (this is expected to happen during shutdown)
Dec 19 02:29:43 [W tensorpipe_agent.cpp:547] RPC agent for worker2 encountered error when reading incoming request from worker1: EOF: end of file (this is expected to happen during shutdown)
Dec 19 02:29:44 ok (2.554s)
Dec 19 02:29:46   test_return_future_remote (__main__.TensorPipeRpcTestWithSpawn) ... [W tensorpipe_agent.cpp:547] RPC agent for worker3 encountered error when reading incoming request from worker2: EOF: end of file (this is expected to happen during shutdown)
Dec 19 02:29:47 ok (2.656s)

See CircleCI build pytorch_linux_backward_compatibility_check_test (4/4)

Step: "Report results" (full log | diagnosis details | 🔁 rerun)

Dec 19 01:28:57 ModuleNotFoundError: No module named 'boto3'
Dec 19 01:28:57 + CIRCLE_BRANCH=gh/jerryzh168/524/head
Dec 19 01:28:57 + export CIRCLE_JOB=pytorch_linux_backward_compatibility_check_test
Dec 19 01:28:57 + CIRCLE_JOB=pytorch_linux_backward_compatibility_check_test
Dec 19 01:28:57 + export CIRCLE_WORKFLOW_ID=539f9711-13dd-49ef-9742-1e2663af2630
Dec 19 01:28:57 + CIRCLE_WORKFLOW_ID=539f9711-13dd-49ef-9742-1e2663af2630
Dec 19 01:28:57 + cd workspace
Dec 19 01:28:57 + python test/print_test_stats.py test
Dec 19 01:28:57 Traceback (most recent call last):
Dec 19 01:28:57   File "test/print_test_stats.py", line 12, in <module>
Dec 19 01:28:57     import boto3
Dec 19 01:28:57 ModuleNotFoundError: No module named 'boto3'


Exited with code exit status 1


1 failure not recognized by patterns:

Job Step Action
CircleCI pytorch_linux_bionic_py3_8_gcc9_coverage_test1 Run tests 🔁 rerun
1 job timed out:
  • pytorch_linux_bionic_py3_8_gcc9_coverage_test2

🚧 1 fixed upstream failure:

These were probably caused by upstream breakages that were already fixed.

Please rebase on the viable/strict branch (expand for instructions)

If your commit is older than viable/strict, run these commands:

git fetch https://github.com/pytorch/pytorch viable/strict
git rebase FETCH_HEAD

Check out the recency history of this "viable master" tracking branch.


This comment was automatically generated by Dr. CI (expand for details).Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions to the (internal) Dr. CI Users group.

This comment has been revised 14 times.

torch/quantization/fx/qconfig_utils.py Outdated Show resolved Hide resolved
…all_method"

Summary:
This adds support to configure qconfig for a call_method, e.g. x.chunk, this will help workaround
a problem in our internal model.

TODO: since call_method is also a string and we flatten the qconfig, might need to resolve namespace conflict between
call_method and module_name
TODO: Add scope support to set the qconfig for call_method correctly with original qconfig

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:

Differential Revision: [D25651828](https://our.internmc.facebook.com/intern/diff/D25651828)

[ghstack-poisoned]
…all_method"

Summary:
This adds support to configure qconfig for a call_method, e.g. x.chunk, this will help workaround
a problem in our internal model.

TODO: since call_method is also a string and we flatten the qconfig, might need to resolve namespace conflict between
call_method and module_name
TODO: Add scope support to set the qconfig for call_method correctly with original qconfig

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:

Differential Revision: [D25651828](https://our.internmc.facebook.com/intern/diff/D25651828)

[ghstack-poisoned]
jerryzh168 added a commit that referenced this pull request Dec 19, 2020
Summary:
This adds support to configure qconfig for a call_method, e.g. x.chunk, this will help workaround
a problem in our internal model.

TODO: since call_method is also a string and we flatten the qconfig, might need to resolve namespace conflict between
call_method and module_name
TODO: Add scope support to set the qconfig for call_method correctly with original qconfig

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:

ghstack-source-id: d1f9111da8fb3db58d08590195f287fcd43a5f3c
Pull Request resolved: #49621
@facebook-github-bot
Copy link
Contributor

This pull request has been merged in 5cde23f.

1 similar comment
@facebook-github-bot
Copy link
Contributor

This pull request has been merged in 5cde23f.

@facebook-github-bot facebook-github-bot deleted the gh/jerryzh168/524/head branch December 22, 2020 15:17
hwangdeyu pushed a commit to hwangdeyu/pytorch that referenced this pull request Jan 6, 2021
…ytorch#49621)

Summary:
Pull Request resolved: pytorch#49621

This adds support to configure qconfig for a call_method, e.g. x.chunk, this will help workaround
a problem in our internal model.

TODO: since call_method is also a string and we flatten the qconfig, might need to resolve namespace conflict between
call_method and module_name
TODO: Add scope support to set the qconfig for call_method correctly with original qconfig

Test Plan: Imported from OSS

Reviewed By: vkuzo

Differential Revision: D25651828

fbshipit-source-id: 82d66b121d37c8274fd481b6a2e9f9b54c5ca73d
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants