Skip to content

Improvements to quantizer: Removed unused qType field, add reshape op#6179

Merged
yufenglee merged 4 commits into
microsoft:masterfrom
pranav-prakash:master
Feb 26, 2021
Merged

Improvements to quantizer: Removed unused qType field, add reshape op#6179
yufenglee merged 4 commits into
microsoft:masterfrom
pranav-prakash:master

Conversation

@pranav-prakash
Copy link
Copy Markdown
Contributor

@pranav-prakash pranav-prakash commented Dec 18, 2020

  • Remove the unused qType field. This was always set to uint8 anyway, which was incorrect for int8 quantization.
  • Fix case when quantized bias already present in quantizer map
  • Support for Reshape op added, which is a passthrough (no-op in terms of quantization).

If bias is shared between multiple nodes and we've already quantized it, just return the quantized name from the map
@pranav-prakash pranav-prakash changed the title Improvements to quantizer: Fix int8 support, add reshape op Improvements to quantizer: Removed unused qType field, add reshape op Feb 22, 2021
Comment thread onnxruntime/python/tools/quantization/operators/reshape.py Outdated
yufenglee
yufenglee previously approved these changes Feb 23, 2021
@yufenglee
Copy link
Copy Markdown
Member

/azp run Linux CPU CI Pipeline,Linux CPU x64 NoContribops CI Pipeline,Linux GPU CI Pipeline,Linux GPU TensorRT CI Pipeline,MacOS CI Pipeline,MacOS NoContribops CI Pipeline,Windows CPU CI Pipeline,Windows GPU CI Pipeline,Windows GPU TensorRT CI Pipeline

@yufenglee
Copy link
Copy Markdown
Member

/azp run orttraining-linux-ci-pipeline,orttraining-mac-ci-pipeline,orttraining-linux-gpu-ci-pipeline,centos7_cpu,Linux CPU Minimal Build E2E CI Pipeline,Linux Nuphar CI Pipeline,MacOS NoContribops CI Pipeline,Linux OpenVINO CI Pipeline,orttraining-distributed

@azure-pipelines
Copy link
Copy Markdown

Azure Pipelines successfully started running 9 pipeline(s).

1 similar comment
@azure-pipelines
Copy link
Copy Markdown

Azure Pipelines successfully started running 9 pipeline(s).

@yufenglee
Copy link
Copy Markdown
Member

@pranav-prakash , it fails some unit tests. Could you please fix them?

@pranav-prakash
Copy link
Copy Markdown
Contributor Author

@yufenglee Fixed. As part of fixing that I saw that in quant_utils the type of qType was inconsistent: in all other places qType is of type onnx_proto.TensorProto, but in quantize_nparray it was expected to be of type QuantType. This meant that there was actually a latent bug in the existing pad.py because in line 42 we had called

quantize_nparray(quantized_input_value.qType, padding_constant_array, scale_value, zp_value)

but then in line 52 we call

pad_value_qnodes = self.quantizer._get_quantize_input_nodes(node, 2, quantized_input_value.qType, ...

and _get_quantize_input_nodes expects qType to be of type onnx_proto.TensorProto.

I've cleaned it up so everything is consistent in using ``onnx_proto.TensorProto`.

@yufenglee
Copy link
Copy Markdown
Member

/azp run Linux CPU CI Pipeline,Linux CPU x64 NoContribops CI Pipeline,Linux GPU CI Pipeline,Linux GPU TensorRT CI Pipeline,MacOS CI Pipeline,MacOS NoContribops CI Pipeline,Windows CPU CI Pipeline,Windows GPU CI Pipeline,Windows GPU TensorRT CI Pipeline

@yufenglee
Copy link
Copy Markdown
Member

/azp run orttraining-linux-ci-pipeline,orttraining-mac-ci-pipeline,orttraining-linux-gpu-ci-pipeline,centos7_cpu,Linux CPU Minimal Build E2E CI Pipeline,Linux Nuphar CI Pipeline,MacOS NoContribops CI Pipeline,Linux OpenVINO CI Pipeline,orttraining-distributed

@azure-pipelines
Copy link
Copy Markdown

Azure Pipelines successfully started running 9 pipeline(s).

1 similar comment
@azure-pipelines
Copy link
Copy Markdown

Azure Pipelines successfully started running 9 pipeline(s).

@yufenglee
Copy link
Copy Markdown
Member

/azp run orttraining-amd-gpu-ci-pipeline

@azure-pipelines
Copy link
Copy Markdown

Azure Pipelines successfully started running 1 pipeline(s).

@yufenglee yufenglee merged commit d517579 into microsoft:master Feb 26, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants