Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fx quant: fix types on _find_quants #49616

Closed
wants to merge 5 commits into from
Closed

Conversation

vkuzo
Copy link
Contributor

@vkuzo vkuzo commented Dec 18, 2020

Stack from ghstack:

Summary:

Add types to _find_quants I/O and fix resulting errors,
needed for an upcoming bug fix.

Test Plan:

mypy torch/quantization
python test/test_quantization.py TestQuantizeFx

Reviewers:

Subscribers:

Tasks:

Tags:

Differential Revision: D25645719

Summary:

Add types to `_find_quants` I/O and fix resulting errors,
needed for an upcoming bug fix.

Test Plan:

```
mypy torch/quantization
python test/test_quantization.py TestQuantizeFx
```

Reviewers:

Subscribers:

Tasks:

Tags:

[ghstack-poisoned]
@facebook-github-bot
Copy link
Contributor

facebook-github-bot commented Dec 18, 2020

💊 CI failures summary and remediations

As of commit 7b80f03 (more details on the Dr. CI page):


  • 2/2 failures possibly* introduced in this PR
    • 1/2 non-CircleCI failure(s)

🕵️ 1 new failure recognized by patterns

The following CI failures do not appear to be due to upstream breakages:

See CircleCI build pytorch_linux_bionic_py3_8_gcc9_coverage_test1 (1/1)

Step: "Run tests" (full log | diagnosis details | 🔁 rerun)

Dec 21 20:00:14 [E request_callback_no_python.cpp:636] Received error while processing request type 258: RuntimeError: Can not pickle torch.futures.Future
Dec 21 20:00:14 At:
Dec 21 20:00:14   /opt/conda/lib/python3.8/site-packages/torch/distributed/rpc/internal.py(120): serialize
Dec 21 20:00:14   /opt/conda/lib/python3.8/site-packages/torch/distributed/rpc/internal.py(172): serialize
Dec 21 20:00:14 
Dec 21 20:00:14 [E request_callback_no_python.cpp:636] Received error while processing request type 258: RuntimeError: Can not pickle torch.futures.Future
Dec 21 20:00:14 
Dec 21 20:00:14 At:
Dec 21 20:00:14   /opt/conda/lib/python3.8/site-packages/torch/distributed/rpc/internal.py(120): serialize
Dec 21 20:00:14   /opt/conda/lib/python3.8/site-packages/torch/distributed/rpc/internal.py(172): serialize
Dec 21 20:00:14 
Dec 21 20:00:14 [E request_callback_no_python.cpp:636] Received error while processing request type 258: RuntimeError: Can not pickle torch.futures.Future
Dec 21 20:00:14 
Dec 21 20:00:14 At:
Dec 21 20:00:14   /opt/conda/lib/python3.8/site-packages/torch/distributed/rpc/internal.py(120): serialize
Dec 21 20:00:14   /opt/conda/lib/python3.8/site-packages/torch/distributed/rpc/internal.py(172): serialize
Dec 21 20:00:14 
Dec 21 20:00:14 [W tensorpipe_agent.cpp:547] RPC agent for worker0 encountered error when reading incoming request from worker1: EOF: end of file (this is expected to happen during shutdown)
Dec 21 20:00:14 [W tensorpipe_agent.cpp:547] RPC agent for worker0 encountered error when reading incoming request from worker2: EOF: end of file (this is expected to happen during shutdown)
Dec 21 20:00:14 [W tensorpipe_agent.cpp:547] RPC agent for worker3 encountered error when reading incoming request from worker2: EOF: end of file (this is expected to happen during shutdown)
Dec 21 20:00:15 ok (2.761s)
Dec 21 20:00:17   test_return_future_remote (__main__.TensorPipeRpcTestWithSpawn) ... [W tensorpipe_agent.cpp:547] RPC agent for worker0 encountered error when reading incoming request from worker2: EOF: end of file (this is expected to happen during shutdown)

1 job timed out:

  • pytorch_linux_bionic_py3_8_gcc9_coverage_test1

This comment was automatically generated by Dr. CI (expand for details).Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions to the (internal) Dr. CI Users group.

This comment has been revised 21 times.

@@ -981,7 +985,7 @@ def visit_arg(arg):
# overwrite previous activation post process constructor if
# necessary
quants[arg.name] = (
DefaultQuantizeHandler(self, arg), act_post_process_ctr)
DefaultQuantizeHandler(self, arg), act_post_process_ctr, is_weight)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we don't need is_weight, this info is used to decide act_post_process_ctr already in L976

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also we can remove L978-979 I think

Copy link
Contributor

@jerryzh168 jerryzh168 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as discussed in chat, we can remove DefaultQuantizeHandler, thanks!

Summary:

Add types to `_find_quants` I/O and fix resulting errors,
needed for an upcoming bug fix.

Test Plan:

```
mypy torch/quantization
python test/test_quantization.py TestQuantizeFx
```

Reviewers:

Subscribers:

Tasks:

Tags:

Differential Revision: [D25645719](https://our.internmc.facebook.com/intern/diff/D25645719)

[ghstack-poisoned]
@@ -571,14 +574,9 @@ def load_non_quantized(n):
return env[n.name]

def load_quantized(n):
if n.name not in quant_env:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this code was flagged unreachable, deleting

Summary:

Add types to `_find_quants` I/O and fix resulting errors,
needed for an upcoming bug fix.

Test Plan:

```
mypy torch/quantization
python test/test_quantization.py TestQuantizeFx
```

Reviewers:

Subscribers:

Tasks:

Tags:

Differential Revision: [D25645719](https://our.internmc.facebook.com/intern/diff/D25645719)

[ghstack-poisoned]
@vkuzo
Copy link
Contributor Author

vkuzo commented Dec 19, 2020

as discussed in chat, we can remove DefaultQuantizeHandler, thanks!

separate PR, to keep things small? since this is really just annotating types and fixing bugs uncovered by that

Summary:

Add types to `_find_quants` I/O and fix resulting errors,
needed for an upcoming bug fix.

Test Plan:

```
mypy torch/quantization
python test/test_quantization.py TestQuantizeFx
```

Reviewers:

Subscribers:

Tasks:

Tags:

Differential Revision: [D25645719](https://our.internmc.facebook.com/intern/diff/D25645719)

[ghstack-poisoned]
Summary:

Add types to `_find_quants` I/O and fix resulting errors,
needed for an upcoming bug fix.

Test Plan:

```
mypy torch/quantization
python test/test_quantization.py TestQuantizeFx
```

Reviewers:

Subscribers:

Tasks:

Tags:

Differential Revision: [D25645719](https://our.internmc.facebook.com/intern/diff/D25645719)

[ghstack-poisoned]
@facebook-github-bot
Copy link
Contributor

This pull request has been merged in edce6b1.

@facebook-github-bot facebook-github-bot deleted the gh/vkuzo/194/head branch December 25, 2020 15:16
hwangdeyu pushed a commit to hwangdeyu/pytorch that referenced this pull request Jan 6, 2021
Summary:
Pull Request resolved: pytorch#49616

Add types to `_find_quants` I/O and fix resulting errors,
needed for an upcoming bug fix.

Test Plan:
```
mypy torch/quantization
python test/test_quantization.py TestQuantizeFx
```

Imported from OSS

Reviewed By: jerryzh168

Differential Revision: D25645719

fbshipit-source-id: 4bf788b55fd4fd086c83a4438b9c2df22b9cff49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants