Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[quantization]: Add support for quantization aware training for Leaky Relu and Sigmoid #45593

Closed
raghuramank100 opened this issue Sep 30, 2020 · 4 comments
Closed

[quantization]: Add support for quantization aware training for Leaky Relu and Sigmoid #45593

raghuramank100 opened this issue Sep 30, 2020 · 4 comments
Assignees
@jerryzh168
Labels
oncall: quantization Quantization support in PyTorch triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module
Projects
Quantization Triage

Comments

@raghuramank100
Copy link
Contributor

raghuramank100 commented Sep 30, 2020 •
edited by pytorch-probot bot

Currently, pytorch supports quantized implementations of leaky relu and sigmoid. For the case of sigmoid, the outputs of the quantized module are quantized over the range [0,1]. For the leaky_relu, the scale and zero-point of the input is used to define the range for the output. This is not modeled during quantization aware training.
A possible solution is to define:

  1. nn.qat.sigmoid module: that has a fake-quantize module at the output with a fixed scale and zero-point.
  2. nn.qat.leaky_relu module: Needs to have a fake-quantize module at the output with scale and zero-point matching that of the input. This may not be possible in eager mode as we do not know the input scale and zero-point, but should be possible with fx quantization.

cc @jerryzh168 @jianyuh @dzhulgakov @raghuramank100 @jamesr66a @vkuzo
@raghuramank100 raghuramank100 added the oncall: quantization Quantization support in PyTorch label Sep 30, 2020
@jerryzh168
Copy link
Contributor

jerryzh168 commented Sep 30, 2020 •
edited

I think for sigmoid we don't need a qat module, it should be supported in the workflow, I'm have WIP PRs starting from #45538

@jerryzh168
Copy link
Contributor

for ops that inherits input quantization parameters (fx graph mode only): #45473
for ops that has fixed quantization parameters: #45538 and #45539

@jerryzh168 jerryzh168 added this to To triage in Quantization Triage Sep 30, 2020
@jerryzh168 jerryzh168 moved this from Need Triage to In Progress in Quantization Triage Sep 30, 2020
jerryzh168 added a commit that referenced this issue Oct 1, 2020
@jerryzh168
[quant] Add quantized::leaky_relu that takes scale/zero_point as input
3c34553 
Summary:
#45593

Previously quantized leaky_relu does not require observation and just inherits
the quantization parameters from input, but that does not work very well in qat
This PR added a quantized::leaky_relu that has observation for output and it will
become the default leaky_relu that our quantization tools produce (eager/graph mode)

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:

[ghstack-poisoned]
@jerryzh168 jerryzh168 mentioned this issue Oct 1, 2020
Closed
jerryzh168 added a commit that referenced this issue Oct 2, 2020
@jerryzh168
Update on "[quant] Add quantized::leaky_relu that takes scale/zero_po…
1d3eb3b 
…int as input"

Summary:
#45593

Previously quantized leaky_relu does not require observation and just inherits
the quantization parameters from input, but that does not work very well in qat
This PR added a quantized::leaky_relu that has observation for output and it will
become the default leaky_relu that our quantization tools produce (eager/graph mode)

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:

Differential Revision: [D24067681](https://our.internmc.facebook.com/intern/diff/D24067681)

[ghstack-poisoned]
jerryzh168 added a commit that referenced this issue Oct 2, 2020
@jerryzh168
Update on "[quant] Add quantized::leaky_relu that takes scale/zero_po…
952a5ce 
…int as input"

Summary:
#45593

Previously quantized leaky_relu does not require observation and just inherits
the quantization parameters from input, but that does not work very well in qat
This PR added a quantized::leaky_relu that has observation for output and it will
become the default leaky_relu that our quantization tools produce (eager/graph mode)

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:

Differential Revision: [D24067681](https://our.internmc.facebook.com/intern/diff/D24067681)

[ghstack-poisoned]
jerryzh168 added a commit that referenced this issue Oct 2, 2020
@jerryzh168
Update on "[quant] Add quantized::leaky_relu that takes scale/zero_po…
19a7faf 
…int as input"

Summary:
#45593

Previously quantized leaky_relu does not require observation and just inherits
the quantization parameters from input, but that does not work very well in qat
This PR added a quantized::leaky_relu that has observation for output and it will
become the default leaky_relu that our quantization tools produce (eager/graph mode)

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:

Differential Revision: [D24067681](https://our.internmc.facebook.com/intern/diff/D24067681)

[ghstack-poisoned]
jerryzh168 added a commit that referenced this issue Oct 5, 2020
@jerryzh168
Update on "[quant] Add quantized::leaky_relu that takes scale/zero_po…
c37f812 
…int as input"

Summary:
#45593

Previously quantized leaky_relu does not require observation and just inherits
the quantization parameters from input, but that does not work very well in qat
This PR added a quantized::leaky_relu that has observation for output and it will
become the default leaky_relu that our quantization tools produce (eager/graph mode)

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:

Differential Revision: [D24067681](https://our.internmc.facebook.com/intern/diff/D24067681)

[ghstack-poisoned]
jerryzh168 added a commit that referenced this issue Oct 6, 2020
@jerryzh168
Update on "[quant] Add quantized::leaky_relu that takes scale/zero_po…
762545a 
…int as input"

Summary:
#45593

Previously quantized leaky_relu does not require observation and just inherits
the quantization parameters from input, but that does not work very well in qat
This PR added a quantized::leaky_relu that has observation for output and it will
become the default leaky_relu that our quantization tools produce (eager/graph mode)

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:

Differential Revision: [D24067681](https://our.internmc.facebook.com/intern/diff/D24067681)

[ghstack-poisoned]
facebook-github-bot pushed a commit that referenced this issue Oct 6, 2020
@jerryzh168 @facebook-github-bot
[quant] Add quantized::leaky_relu that takes scale/zero_point as input (
d1fc155 
#45702)

Summary:
Pull Request resolved: #45702

#45593

Previously quantized leaky_relu does not require observation and just inherits
the quantization parameters from input, but that does not work very well in qat
This PR added a quantized::leaky_relu that has observation for output and it will
become the default leaky_relu that our quantization tools produce (eager/graph mode)

Test Plan: Imported from OSS

Reviewed By: raghuramank100

Differential Revision: D24067681

fbshipit-source-id: d216738344363794b82bd3d75c8587a4b9415bca
@raghuramank100
Copy link
Contributor Author

Can we close this issue now?

@raghuramank100 raghuramank100 added the triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module label Mar 8, 2021
@jerryzh168
Copy link
Contributor

I think so

Quantization Triage automation moved this from In Progress to Done Mar 8, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jerryzh168 jerryzh168

Labels
oncall: quantization Quantization support in PyTorch triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module
Projects
Quantization Triage
  
Done
Milestone
No milestone
Development

No branches or pull requests
2 participants
@jerryzh168 @raghuramank100