[quant][fx] Add support for GRU in fx graph mode quantization #91976
Conversation
Summary: might be needed by a meta-internal use case Test Plan: python test/test_quantization.py TestQuantizeFxOps.test_rnn Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned]
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/91976
Note: Links to docs will display an error until the docs builds have been completed. ✅ No FailuresAs of commit 83f1dfd: This comment was automatically generated by Dr. CI and updates every 15 minutes. |
Summary: might be needed by a meta-internal use case Test Plan: python test/test_quantization.py TestQuantizeFxOps.test_rnn Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: febc85f4ebb0744c7f7d54d4e7f6a212b3a699db Pull Request resolved: #91976
| if wn.startswith("weight"): | ||
| weight_or_bias = get_quantized_weight(self, wn) | ||
| else: | ||
| weight_or_bias = getattr(self, wn) |
There was a problem hiding this comment.
just curious - is it customary to not quantize bias at all?
There was a problem hiding this comment.
it's a design choice when we are designing eager mode quantization. in computation, bias is actually quantized to int32 with the scale of input and weight, but we only have this information in graph mode quantization, not in eager, so we have to keep bias fp32 and quantize it internally in the quantized linear operator
| else: | ||
| result = _VF.gru(input, batch_sizes, hx, self.get_flat_weights(), self.bias, | ||
| self.num_layers, self.dropout, self.training, self.bidirectional) | ||
| output = result[0] |
There was a problem hiding this comment.
nit: does output, hidden = result work here?
There was a problem hiding this comment.
probably, this is just copied from forward function of GRU though, will add a comment in the beginning
…ion" Summary: might be needed by a meta-internal use case Test Plan: python test/test_quantization.py TestQuantizeFxOps.test_rnn Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned]
Summary: might be needed by a meta-internal use case Test Plan: python test/test_quantization.py TestQuantizeFxOps.test_rnn Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 9bb29827fb27de525b869a24702024e688ae22d7 Pull Request resolved: #91976
|
@pytorchbot merge -g |
Merge startedYour change will be merged once all checks on your PR pass since you used the green (-g) flag (ETA: 0-4 Hours). Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
Merge failedReason: The following mandatory check(s) failed (Rule Dig deeper by viewing the failures on hud Details for Dev Infra teamRaised by workflow job |
|
@pytorchbot rebase |
|
@pytorchbot successfully started a rebase job. Check the current status here |
…ion" Summary: might be needed by a meta-internal use case Test Plan: python test/test_quantization.py TestQuantizeFxOps.test_rnn Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned]
|
Successfully rebased |
Summary: might be needed by a meta-internal use case Test Plan: python test/test_quantization.py TestQuantizeFxOps.test_rnn Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 2d3e596f7afa5751e3c28e193b46d9ac43c7aa03 Pull Request resolved: #91976
…ion" Summary: might be needed by a meta-internal use case Test Plan: python test/test_quantization.py TestQuantizeFxOps.test_rnn Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned]
Summary: might be needed by a meta-internal use case Test Plan: python test/test_quantization.py TestQuantizeFxOps.test_rnn Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: ca7ed35362cc1ec34f179bacaa0dd9fe9f69f2e8 Pull Request resolved: #91976
|
@pytorchbot merge -g |
Merge startedYour change will be merged once all checks on your PR pass since you used the green (-g) flag (ETA: 0-4 Hours). Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
Stack from ghstack (oldest at bottom):
Summary:
might be needed by a meta-internal use case
Test Plan:
python test/test_quantization.py TestQuantizeFxOps.test_rnn
Reviewers:
Subscribers:
Tasks:
Tags: