[quant] Add quantized::leaky_relu that takes scale/zero_point as input #45702

jerryzh168 · 2020-10-01T23:38:33Z

Stack from ghstack:

[quant][fx][graphmode] Add support for sigmoid #45885 [quant][fx][graphmode] Add support for sigmoid
[quant][eagermode] Add support for sigmoid module #45884 [quant][eagermode] Add support for sigmoid module
[quant] Add quantized Sigmoid module #45883 [quant] Add quantized Sigmoid module
[quant] Add quantized::sigmoid that take output_scale/output_zero_point as input #45882 [quant] Add quantized::sigmoid that take output_scale/output_zero_point as input
[quant][graphmode][fx] Add warning for unsupported case #45714 [quant][graphmode][fx] Add warning for unsupported case
[quant][graphmode][fx][eagermode] Add leaky relu support in quantization workflows #45712 [quant][graphmode][fx][eagermode] Add leaky relu support in quantization workflows
[quant] Add quantized LeakyReLU module #45711 [quant] Add quantized LeakyReLU module
[quant] Add quantized::leaky_relu that takes scale/zero_point as input #45702 [quant] Add quantized::leaky_relu that takes scale/zero_point as input

Summary:
#45593

Previously quantized leaky_relu does not require observation and just inherits
the quantization parameters from input, but that does not work very well in qat
This PR added a quantized::leaky_relu that has observation for output and it will
become the default leaky_relu that our quantization tools produce (eager/graph mode)

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:

Differential Revision: D24067681

Summary: #45593 Previously quantized leaky_relu does not require observation and just inherits the quantization parameters from input, but that does not work very well in qat This PR added a quantized::leaky_relu that has observation for output and it will become the default leaky_relu that our quantization tools produce (eager/graph mode) Test Plan: Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned]

…int as input" Summary: #45593 Previously quantized leaky_relu does not require observation and just inherits the quantization parameters from input, but that does not work very well in qat This PR added a quantized::leaky_relu that has observation for output and it will become the default leaky_relu that our quantization tools produce (eager/graph mode) Test Plan: Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D24067681](https://our.internmc.facebook.com/intern/diff/D24067681) [ghstack-poisoned]

aten/src/ATen/native/quantized/cpu/qrelu.cpp

raghuramank100 · 2020-10-02T04:55:16Z

aten/src/ATen/native/quantized/cpu/qrelu.cpp

@@ -113,7 +113,7 @@ Tensor& leaky_relu_out_quantized_cpu(Tensor& result, const Tensor& self,
  return result;
 }

-Tensor heaky_relu_quantized_cpu(const Tensor& self, Scalar negval) {
+Tensor leaky_relu_quantized_cpu(const Tensor& self, Scalar negval) {


For my understanding: When is this called compared to QLeakyRelu::run?

this is called by the original leaky_relu op that works on both float and quantized tensors and it does not need quantization parameter, QLeakyRelu::run is called by quantized::leaky_relu, which only works on quantized tensor and needs output quantization parameters

…int as input" Summary: #45593 Previously quantized leaky_relu does not require observation and just inherits the quantization parameters from input, but that does not work very well in qat This PR added a quantized::leaky_relu that has observation for output and it will become the default leaky_relu that our quantization tools produce (eager/graph mode) Test Plan: Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D24067681](https://our.internmc.facebook.com/intern/diff/D24067681) [ghstack-poisoned]

codecov · 2020-10-05T21:59:47Z

Codecov Report

Merging #45702 into gh/jerryzh168/448/base will not change coverage.
The diff coverage is n/a.

@@                   Coverage Diff                   @@
##           gh/jerryzh168/448/base   #45702   +/-   ##
=======================================================
  Coverage                   68.32%   68.32%           
=======================================================
  Files                         410      410           
  Lines                       52981    52981           
=======================================================
  Hits                        36200    36200           
  Misses                      16781    16781

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 162717e...762545a. Read the comment docs.

…int as input" Summary: #45593 Previously quantized leaky_relu does not require observation and just inherits the quantization parameters from input, but that does not work very well in qat This PR added a quantized::leaky_relu that has observation for output and it will become the default leaky_relu that our quantization tools produce (eager/graph mode) Test Plan: Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D24067681](https://our.internmc.facebook.com/intern/diff/D24067681) [ghstack-poisoned]

…nt as input Summary: Same changes as the stack for leaky_relu: #45702 Test Plan: Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned]

facebook-github-bot · 2020-10-06T18:19:38Z

This pull request has been merged in d1fc155.

…ut_zero_point as input" Summary: Same changes as the stack for leaky_relu: #45702 Test Plan: Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D24129113](https://our.internmc.facebook.com/intern/diff/D24129113) [ghstack-poisoned]

…utput_scale/output_zero_point as input" Summary: Same changes as the stack for leaky_relu: #45702 Test Plan: Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D24129113](https://our.internmc.facebook.com/intern/diff/D24129113) [ghstack-poisoned]

…ut_zero_point as input" Summary: Same changes as the stack for leaky_relu: #45702 Test Plan: Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D24129113](https://our.internmc.facebook.com/intern/diff/D24129113) [ghstack-poisoned]

…nt as input (#45882) Summary: Pull Request resolved: #45882 Same changes as the stack for leaky_relu: #45702 Test Plan: Imported from OSS Reviewed By: z-a-f Differential Revision: D24129113 fbshipit-source-id: a26da33f877d3bdeea1976b69b2bd9369c2bf196

This was referenced Oct 2, 2020

[quant] Add quantized LeakyReLU module #45711

Closed

[quant][graphmode][fx][eagermode] Add leaky relu support in quantization workflows #45712

Closed

jerryzh168 mentioned this pull request Oct 2, 2020

[quant][graphmode][fx] Add warning for unsupported case #45714

Closed

raghuramank100 reviewed Oct 2, 2020

View reviewed changes

aten/src/ATen/native/quantized/cpu/qrelu.cpp Show resolved Hide resolved

raghuramank100 reviewed Oct 2, 2020

View reviewed changes

raghuramank100 approved these changes Oct 2, 2020

View reviewed changes

jerryzh168 mentioned this pull request Oct 6, 2020

[quant] Add quantized::sigmoid that take output_scale/output_zero_point as input #45882

Closed

jerryzh168 added a commit that referenced this pull request Oct 6, 2020

[quant] Add quantized::sigmoid that take output_scale/output_zero_poi…

c0b9d44

…nt as input Summary: Same changes as the stack for leaky_relu: #45702 Test Plan: Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned]

This was referenced Oct 6, 2020

[quant] Add quantized Sigmoid module #45883

Closed

[quant][eagermode] Add support for sigmoid module #45884

Closed

[quant][fx][graphmode] Add support for sigmoid #45885

Closed

facebook-github-bot closed this in d1fc155 Oct 6, 2020

facebook-github-bot added the Merged label Oct 6, 2020

facebook-github-bot deleted the gh/jerryzh168/448/head branch October 10, 2020 14:16

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[quant] Add quantized::leaky_relu that takes scale/zero_point as input #45702

[quant] Add quantized::leaky_relu that takes scale/zero_point as input #45702

jerryzh168 commented Oct 1, 2020 •

edited

raghuramank100 Oct 2, 2020

jerryzh168 Oct 5, 2020

codecov bot commented Oct 5, 2020 •

edited

facebook-github-bot commented Oct 6, 2020

[quant] Add quantized::leaky_relu that takes scale/zero_point as input #45702

[quant] Add quantized::leaky_relu that takes scale/zero_point as input #45702

Conversation

jerryzh168 commented Oct 1, 2020 • edited

raghuramank100 Oct 2, 2020

Choose a reason for hiding this comment

jerryzh168 Oct 5, 2020

Choose a reason for hiding this comment

codecov bot commented Oct 5, 2020 • edited

Codecov Report

facebook-github-bot commented Oct 6, 2020

jerryzh168 commented Oct 1, 2020 •

edited

codecov bot commented Oct 5, 2020 •

edited