Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

能否支持权重和激活位宽不同的量化? #559

Open
frankgt opened this issue May 15, 2024 · 0 comments
Open

能否支持权重和激活位宽不同的量化? #559

frankgt opened this issue May 15, 2024 · 0 comments

Comments

@frankgt
Copy link

frankgt commented May 15, 2024

ppq是一个很棒的框架,非常系统的考虑到了模型量化落地的方方面面,非常值得学习。
尝试了下量化效果的确不错,但是有一个问题,目前ppq能否支持权重和激活的bit位宽不同的量化呢?
比如a16w8,即激活16bit,权重8bit。
初步看了下相关的代码(ppq/executor/torch.py, L: 515),目前似乎权重和激活是一起处理的,没有进行区分。
`

if operation is an QuantableOperation, we have to quant its inputs and outputs at first.

            if isinstance(operation, QuantableOperation):
                input_configs = [_ for _ in operation.config.input_quantization_config]
                inputs = [self.quantize_function(input, config) for input, config in zip(inputs, input_configs)]

`

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant