Skip to content
This repository was archived by the owner on Jun 3, 2025. It is now read-only.

Conversation

@anmarques
Copy link
Member

Changes in quantization:

  1. Created flag to remove Q/DQ from output activations for conv layers. Set to True by default.
  2. Changed default to True for flag to remove Q/DQ from output activations for linear layers.
  3. Created flag to change number of bits for weight activation (similar to the already existing flag for activation bits).
  4. Created flag to avoid quantization of BatchNorm layers when fusing is not used. Set to True by default.
  5. Created flag to enable quantization compatible with tensorrt. Set to False by default.
    • Sets activation quantization to symmetric and UINT8.
    • Sets default fusing function to 'no_fuse'.
  6. Created a wrapper module (BNWrapper) that allows one to freeze BatchNorm statistics when BN layers are not fused. PyTorch default only works when BN layers are fused.
  7. Crated a wrapper module (_AddReLU) for the FloatFunctional class that performs the Add + ReLU operations in ResNet. This wrapper enables the module to be wrapped by a QATWrapper when quantized, and is set to quantize only the first input.

anmarques added 30 commits March 9, 2022 12:14
… range to get_qat_config_config where it has full information about data type.
… range to get_qat_config_config where it has full information about data type.
@bfineran
Copy link
Contributor

bfineran commented Apr 8, 2022

merged in fixes for testing and additional tests from #685

@bfineran bfineran dismissed their stale review April 8, 2022 18:20

contributor

markurtz
markurtz previously approved these changes Apr 8, 2022
@bfineran bfineran requested review from a team, KSGulin and rahul-tuli and removed request for a team April 8, 2022 18:38
spacemanidol
spacemanidol previously approved these changes Apr 8, 2022
@bfineran bfineran dismissed stale reviews from spacemanidol and markurtz via 5f74e31 April 8, 2022 19:12
markurtz
markurtz previously approved these changes Apr 8, 2022
rahul-tuli
rahul-tuli previously approved these changes Apr 8, 2022
@bfineran bfineran dismissed stale reviews from rahul-tuli and markurtz via 91243ff April 8, 2022 19:31
@anmarques anmarques merged commit 1db70be into main Apr 8, 2022
@anmarques anmarques deleted the quantization-refactor branch April 8, 2022 19:44
Copy link
Contributor

@spacemanidol spacemanidol left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

look good

dbogunowicz pushed a commit that referenced this pull request Apr 11, 2022
* Removed output quantization from conv layers

* Added _Add_ReLU module that enables QATWrapper for quantizaiton.

* Removed quantization of output for linear and conv layers by default. Removed fusing of BN and ReLU by default.

* Minor fixes. Style and quality fixes.

* Added support to freezing bn stats.

* Added mode argument to wrapping of train function in BNWrapper

* Set BN fusing back as default.

* Set BN fusing back as default.

* Fixed custom freeze_bn_stats.

* Temporary files for evaluating changes to graphs.

* Added support to tensorrt flag. Moved the computation of quantization range to get_qat_config_config where it has full information about data type.

* Added support to TensorRT quantization

* Included check to account for when weight_qconfig_kwatgs is None.

* Removed output quantization from conv layers

* Added _Add_ReLU module that enables QATWrapper for quantizaiton.

* Removed quantization of output for linear and conv layers by default. Removed fusing of BN and ReLU by default.

* Minor fixes. Style and quality fixes.

* Added support to freezing bn stats.

* Added mode argument to wrapping of train function in BNWrapper

* Set BN fusing back as default.

* Set BN fusing back as default.

* Fixed custom freeze_bn_stats.

* Temporary files for evaluating changes to graphs.

* Added support to tensorrt flag. Moved the computation of quantization range to get_qat_config_config where it has full information about data type.

* Added support to TensorRT quantization

* Included check to account for when weight_qconfig_kwatgs is None.

* Modified argument names for backwards compatibility.

* Updated documentation to reflect changes.

* Updated documentation to reflect changes.

* Updated documentation to reflect changes.

* Fixed default weights data type.

* Style and quality fixes.

* Removed unused method

* Removed output quantization from conv layers

* Added _Add_ReLU module that enables QATWrapper for quantizaiton.

* Removed quantization of output for linear and conv layers by default. Removed fusing of BN and ReLU by default.

* Minor fixes. Style and quality fixes.

* Added support to freezing bn stats.

* Added mode argument to wrapping of train function in BNWrapper

* Set BN fusing back as default.

* Set BN fusing back as default.

* Fixed custom freeze_bn_stats.

* Temporary files for evaluating changes to graphs.

* Added support to tensorrt flag. Moved the computation of quantization range to get_qat_config_config where it has full information about data type.

* Added support to TensorRT quantization

* Included check to account for when weight_qconfig_kwatgs is None.

* Removed output quantization from conv layers

* Added _Add_ReLU module that enables QATWrapper for quantization.

* Removed quantization of output for linear and conv layers by default. Removed fusing of BN and ReLU by default.

* Minor fixes. Style and quality fixes.

* Added support to freezing bn stats.

* Added mode argument to wrapping of train function in BNWrapper

* Set BN fusing back as default.

* Set BN fusing back as default.

* Fixed custom freeze_bn_stats.

* Temporary files for evaluating changes to graphs.

* Added support to tensorrt flag. Moved the computation of quantization range to get_qat_config_config where it has full information about data type.

* Added support to TensorRT quantization

* Included check to account for when weight_qconfig_kwatgs is None.

* Modified argument names for backwards compatibility.

* Updated documentation to reflect changes.

* Updated documentation to reflect changes.

* Updated documentation to reflect changes.

* Fixed default weights data type.

* Style and quality fixes.

* Removed unused method

* Removed output quantization from conv layers

* Added _Add_ReLU module that enables QATWrapper for quantizaiton.

* Removed quantization of output for linear and conv layers by default. Removed fusing of BN and ReLU by default.

* Minor fixes. Style and quality fixes.

* Added support to freezing bn stats.

* Added mode argument to wrapping of train function in BNWrapper

* Set BN fusing back as default.

* Set BN fusing back as default.

* Fixed custom freeze_bn_stats.

* Temporary files for evaluating changes to graphs.

* Added support to tensorrt flag. Moved the computation of quantization range to get_qat_config_config where it has full information about data type.

* Added support to TensorRT quantization

* Included check to account for when weight_qconfig_kwatgs is None.

* Removed output quantization from conv layers

* Added _Add_ReLU module that enables QATWrapper for quantization.

* Removed quantization of output for linear and conv layers by default. Removed fusing of BN and ReLU by default.

* Minor fixes. Style and quality fixes.

* Added support to freezing bn stats.

* Added mode argument to wrapping of train function in BNWrapper

* Set BN fusing back as default.

* Set BN fusing back as default.

* Fixed custom freeze_bn_stats.

* Temporary files for evaluating changes to graphs.

* Added support to tensorrt flag. Moved the computation of quantization range to get_qat_config_config where it has full information about data type.

* Added support to TensorRT quantization

* Included check to account for when weight_qconfig_kwatgs is None.

* Modified argument names for backwards compatibility.

* Updated documentation to reflect changes.

* Updated documentation to reflect changes.

* Updated documentation to reflect changes.

* Fixed default weights data type.

* Style and quality fixes.

* Removed unused method

* Removed output quantization from conv layers

* Added _Add_ReLU module that enables QATWrapper for quantizaiton.

* Removed quantization of output for linear and conv layers by default. Removed fusing of BN and ReLU by default.

* Minor fixes. Style and quality fixes.

* Added support to freezing bn stats.

* Added mode argument to wrapping of train function in BNWrapper

* Set BN fusing back as default.

* Set BN fusing back as default.

* Fixed custom freeze_bn_stats.

* Temporary files for evaluating changes to graphs.

* Added support to tensorrt flag. Moved the computation of quantization range to get_qat_config_config where it has full information about data type.

* Added support to TensorRT quantization

* Included check to account for when weight_qconfig_kwatgs is None.

* Removed output quantization from conv layers

* Added _Add_ReLU module that enables QATWrapper for quantizaiton.

* Removed quantization of output for linear and conv layers by default. Removed fusing of BN and ReLU by default.

* Minor fixes. Style and quality fixes.

* Added support to freezing bn stats.

* Added mode argument to wrapping of train function in BNWrapper

* Set BN fusing back as default.

* Set BN fusing back as default.

* Fixed custom freeze_bn_stats.

* Temporary files for evaluating changes to graphs.

* Added support to tensorrt flag. Moved the computation of quantization range to get_qat_config_config where it has full information about data type.

* Added support to TensorRT quantization

* Included check to account for when weight_qconfig_kwatgs is None.

* Modified argument names for backwards compatibility.

* Updated documentation to reflect changes.

* Updated documentation to reflect changes.

* Updated documentation to reflect changes.

* Fixed default weights data type.

* Style and quality fixes.

* Removed unused method

* Removed testing files

* Style and quality fixes.

* Changed call to get_qat_qconfig to not specify symmetry and data type arguments for default case.

* Removed output quantization from conv layers

* Added _Add_ReLU module that enables QATWrapper for quantizaiton.

* Removed quantization of output for linear and conv layers by default. Removed fusing of BN and ReLU by default.

* Minor fixes. Style and quality fixes.

* Added support to freezing bn stats.

* Added mode argument to wrapping of train function in BNWrapper

* Set BN fusing back as default.

* Set BN fusing back as default.

* Fixed custom freeze_bn_stats.

* Temporary files for evaluating changes to graphs.

* Added support to tensorrt flag. Moved the computation of quantization range to get_qat_config_config where it has full information about data type.

* Added support to TensorRT quantization

* Included check to account for when weight_qconfig_kwatgs is None.

* Removed output quantization from conv layers

* Added _Add_ReLU module that enables QATWrapper for quantization.

* Removed quantization of output for linear and conv layers by default. Removed fusing of BN and ReLU by default.

* Minor fixes. Style and quality fixes.

* Added support to freezing bn stats.

* Added mode argument to wrapping of train function in BNWrapper

* Set BN fusing back as default.

* Set BN fusing back as default.

* Fixed custom freeze_bn_stats.

* Temporary files for evaluating changes to graphs.

* Added support to tensorrt flag. Moved the computation of quantization range to get_qat_config_config where it has full information about data type.

* Added support to TensorRT quantization

* Included check to account for when weight_qconfig_kwatgs is None.

* Modified argument names for backwards compatibility.

* Updated documentation to reflect changes.

* Updated documentation to reflect changes.

* Updated documentation to reflect changes.

* Fixed default weights data type.

* Style and quality fixes.

* Removed unused method

* Removed output quantization from conv layers

* Added _Add_ReLU module that enables QATWrapper for quantizaiton.

* Removed quantization of output for linear and conv layers by default. Removed fusing of BN and ReLU by default.

* Minor fixes. Style and quality fixes.

* Added support to freezing bn stats.

* Added mode argument to wrapping of train function in BNWrapper

* Set BN fusing back as default.

* Set BN fusing back as default.

* Fixed custom freeze_bn_stats.

* Temporary files for evaluating changes to graphs.

* Added support to tensorrt flag. Moved the computation of quantization range to get_qat_config_config where it has full information about data type.

* Added support to TensorRT quantization

* Included check to account for when weight_qconfig_kwatgs is None.

* Removed output quantization from conv layers

* Added _Add_ReLU module that enables QATWrapper for quantizaiton.

* Removed quantization of output for linear and conv layers by default. Removed fusing of BN and ReLU by default.

* Minor fixes. Style and quality fixes.

* Added support to freezing bn stats.

* Added mode argument to wrapping of train function in BNWrapper

* Set BN fusing back as default.

* Set BN fusing back as default.

* Fixed custom freeze_bn_stats.

* Temporary files for evaluating changes to graphs.

* Added support to tensorrt flag. Moved the computation of quantization range to get_qat_config_config where it has full information about data type.

* Added support to TensorRT quantization

* Included check to account for when weight_qconfig_kwatgs is None.

* Modified argument names for backwards compatibility.

* Updated documentation to reflect changes.

* Updated documentation to reflect changes.

* Updated documentation to reflect changes.

* Fixed default weights data type.

* Style and quality fixes.

* Removed unused method

* Removed testing files

* Style and quality fixes.

* Changed call to get_qat_qconfig to not specify symmetry and data type arguments for default case.

* Changed default number of activation and weight bits from None to 8.

* Revert "Changed default number of activation and weight bits from None to 8."

This reverts commit 95e966ed929fa3512331a73667d5ba2ac3d594b1.

* Revert "Changed call to get_qat_qconfig to not specify symmetry and data type arguments for default case."

This reverts commit a675813.

* Lumped qconfig properties into a dataclass.

* Lumped qconfig properties into a dataclass.

* Lumped qconfig properties into a dataclass.

* Resetting conv and linear activation flags to True.

* Renamed class BNWrapper as _BNWrapper.

* Added logging messages for when tensorrt forces overriding of configs.

* Style and quality fixes.

* ConvInteger quantization conversion for quant refactor (#644)

* ConvInteger quantization conversion for quant refactor

* [quantization-refactor] mark/propagate conv export mode (#672)

* batch norm fold with existing bias param bug fix

* Quantization Refactor Tests (#685)

* rebase import fix

* update manager serialization test cases for new quantization params

Co-authored-by: Benjamin Fineran <bfineran@users.noreply.github.com>
Co-authored-by: spacemanidol <dcampos3@illinois.edu>
Co-authored-by: Benjamin <ben@neuralmagic.com>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

0.12 release Pull request pending for 0.12 release.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants