eager quant: remove fake_quant after add/mul nodes during QAT

[vkuzo]

Stack from ghstack:

• fx quant: make sure observer is inserted before a quantized output #49420 fx quant: make sure observer is inserted before a quantized output
• fx quant: fix fq when input is quantized and node does not need fq #49382 fx quant: fix fq when input is quantized and node does not need fq
• fx quant: do not insert observers at quantized inputs #49239 fx quant: do not insert observers at quantized inputs
• fx quant: move {input|output}_quantized_idxs cfg from convert to prepare #49238 fx quant: move {input|output}_quantized_idxs cfg from convert to prepare
• eager quant: remove fake_quant after add/mul nodes during QAT #49213 eager quant: remove fake_quant after add/mul nodes during QAT

Summary:

Changes behavior of Eager mode quantization to remove observation after add_scalar/mul_scalar.
This is not used, and it removes one difference between Eager and FX modes.

Test Plan:

python test/test_quantization.py TestQuantizeFxOps.test_quantized_add_qat
python test/test_quantization.py TestQuantizeFxOps.test_quantized_mul_qat
python test/test_quantization.py TestQuantizationAwareTraining.test_add_scalar_uses_input_qparams
python test/test_quantization.py TestQuantizationAwareTraining.test_mul_scalar_uses_input_qparams

Reviewers:

Subscribers:

Tasks:

Tags:

Differential Revision: D25486276

[facebook-github-bot]

💊 CI failures summary and remediations

As of commit a620fe7 (more details on the Dr. CI page):

---

• 1/1 failures possibly* introduced in this PR
  • 1/1 non-CircleCI failure(s)

---

This comment was automatically generated by Dr. CI (expand for details).

Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions to the (internal) Dr. CI Users group.

This comment has been revised 22 times.

[jerryzh168]

I think we'll need to use the same fake quantize module as the input here to properly simulate quantization since output of quantized::add with scalar input inherits quantization parameters from input

[raghuramank100]

Not sure I follow: The current logic inserts observers as long as one of the inputs is a tensor. why is this a problem for QAT?

[jerryzh168]

we can match eager now and improve later. currently eager mode is incorrect, because the fake quantize is ignored, and the model before and after convert for the qat model would have different numerics. output of add_scalar/mul_scalar need to use the same fake quantize module as input so that the numerics of fake quantize matches the quantized op.

[raghuramank100]

Are the inputs and outputs quantized? In this case, shouldnt we have 5 fake-quants? (add, conv1, add-relu, conv2, output)?

[jerryzh168]

output of add/input of conv1 is considered to be observed here I think

[jerryzh168]

the ideal state would be the output of add/input of conv1 being fake quantized with the same fq module as the input of add.

[jerryzh168]

looks like this breaks some of the numeric_suite tests. we may need to keep the activation_post_process attribute (and just use Identity).

[vkuzo]

looks like this breaks some of the numeric_suite tests. we may need to keep the activation_post_process attribute (and just use Identity).

thanks, the fix was to do the same change on QFunctional. add_scalar and mul_scalar will no longer be supported by Eager NS after this PR, which is intended as Eager NS only works on activations. Theoretically someone could hack it back in with a module swap, but that should probably be outside of this PR.

[facebook-github-bot]

This pull request has been merged in 36b2092.