[quant][graphmode][fx] Merge all quantization mode

[jerryzh168]

Stack from ghstack:

• [quant][graphmode][fx][eagermode] Support sigmoid/hardsigmoid/tanh in eager and fx graph mode #45539 [quant][graphmode][fx][eagermode] Support sigmoid/hardsigmoid/tanh in eager and fx graph mode
• [quant] Add FixedQParamsFakeQuantize module #45538 [quant] Add FixedQParamsFakeQuantize module
• [quant][graphmode][fx][fix] Fix observer insert logic for ops that inherits quantization parameters for input #45473 [quant][graphmode][fx][fix] Fix observer insert logic for ops that inherits quantization parameters for input
• [quant][graphmode][fx] Merge all quantization mode #45292 [quant][graphmode][fx] Merge all quantization mode
• [quant] Use PlaceholderObserver as default dynamic quant observer #45343 [quant] Use PlaceholderObserver as default dynamic quant observer

Summary:
This PR merges all quantization mode and will only expose the following top level functions:

prepare_fx
prepare_qat_fx
convert_fx

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:

Differential Revision: D23913105

[dr-ci]

💊 CI failures summary and remediations

As of commit 1d191e0 (more details on the Dr. CI page):

---

Commit 1d191e0 was recently pushed. Waiting for builds...

---

This comment was automatically generated by Dr. CI (expand for details).

Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions on the GitHub issue tracker or post in the (internal) Dr. CI Users group.

See how this bot performed.

This comment has been revised 109 times.

[vkuzo]

looks awesome, the new names are much easier to read!

[vkuzo]

looks super clean, thanks!

[supriyar]

Can we use activation_is_dynamically_quantized instead here for better readability?

[jerryzh168]

I'm planning to remove that check since it checks for both dynamic and weight only dtypes.

[supriyar]

Not sure I follow. What is the check we will use for dynamic quant in that case?

[jerryzh168]

activation_is_dynamically_quantized checks for float32 dtype which can be both dynamic or weight only quantization.

[supriyar]

Will we also be updating the top level APIs to remove the quantize_{static, dynamic}_fx calls ? https://github.com/pytorch/pytorch/blob/master/torch/quantization/quantize_fx.py#L108-L218

[jerryzh168]

Will we also be updating the top level APIs to remove the quantize_{static, dynamic}_fx calls ? https://github.com/pytorch/pytorch/blob/master/torch/quantization/quantize_fx.py#L108-L218

yes will do that, I'm waiting for previous PRs to be landed to avoid conflicts.

[facebook-github-bot]

This pull request has been merged in ffcb098.

[t-vi]

This seems to break the CI on ROCm with an error that looks like it is this patch:
https://ci.pytorch.org/jenkins/job/pytorch-builds/job/pytorch-linux-bionic-rocm3.7-py3.6-test2/5653//console

FAIL: test_qat_prepare_device_affinity (quantization.test_quantize_fx.TestQuantizeFx)
06:50:29 ----------------------------------------------------------------------
06:50:29 Traceback (most recent call last):
06:50:29   File "/var/lib/jenkins/.local/lib/python3.6/site-packages/torch/testing/_internal/common_quantized.py", line 122, in test_fn
06:50:29     qfunction(*args, **kwargs)
06:50:29   File "/var/lib/jenkins/workspace/test/quantization/test_quantize_fx.py", line 268, in test_qat_prepare_device_affinity
06:50:29     model = prepare_fx(model, qconfig_dict)
06:50:29   File "/var/lib/jenkins/.local/lib/python3.6/site-packages/torch/quantization/quantize_fx.py", line 127, in prepare_fx
06:50:29     'eval mode'
06:50:29 AssertionError: prepare_fx only works for models ineval mode
06:50:29 

Also appreas to break pytorch_linux_xenial_cuda9_2_cudnn7_py3_gcc7_test with the same thing.

[mruberry]

Unlanding. @t-vi nailed it.