Assert all ptq-common bit widths are positive integers #931
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ould be positive integers.
Giuseppe5
approved these changes
Apr 10, 2024
There was a problem hiding this comment.
I was about to suggest somehing like this to fix the issue.
Going a bit deeper, bit width requirements can be further specified, for example bias, weight, and activation bit width for integer quantization can only be greater than 3, since for Ternary and Binary quantization we need a different setup.
For floating point quantization, we can have also 1 bit for mantissa or exponent.
However these checks should probably live somewhere else in the code (e.g., the quantizer should verify that it's valid), so for now this assert should be sufficient.
|
Can you run the pre-commit hooks please? |
Yep, have done so now! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In testing I found that
ptq_common.pyquantize_modelbit widths can take negative and zero values. In the case of zero values, the model outputs NaNs, and in the case of negative values it is a bit more complicated (real values get outputted), but it is still undesired behavior.This PR adds a constraint that all 'ptq_common/quantize_model' bit widths should be positive integers.