Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

quantized results description #39

Open
burui11087 opened this issue Dec 2, 2019 · 7 comments
Open

quantized results description #39

burui11087 opened this issue Dec 2, 2019 · 7 comments
Assignees

Comments

@burui11087
Copy link

burui11087 commented Dec 2, 2019

Thanks for your great work in advance!

I have some questions about the following description:

<p>Quantized模型是对原模型进行转换过程中,将float参数转化为uint8类型,进而产生的模型会更小、运行更快,但是精度会有所下降。</p>

In my daily experiments, I find that the quantized results(accuracy) can go down as well as up when task is classification. So I think it's better to modify this description.

Thanks again.

@dpinthinker
Copy link
Collaborator

@burui11087 Could you tell me on which condition the quantized size will be bigger than the original one?

@dpinthinker
Copy link
Collaborator

Post-training quantization is a conversion technique that can reduce model size while also improving CPU and hardware accelerator latency, with little degradation in model accuracy.
From https://www.tensorflow.org/lite/performance/post_training_quantization

@burui11087
Copy link
Author

Post-training quantization is a conversion technique that can reduce model size while also improving CPU and hardware accelerator latency, with little degradation in model accuracy.
From https://www.tensorflow.org/lite/performance/post_training_quantization

I am not a researcher in the direction of model quantization. At first I thought the official description you posted was correct, but after a lot of experiments( a classification task), I found the quantized results could go down as well as up(I can't post my experiment results here because of corporate data security). After discuss with my colleague, we think the quantization operation can be treated as reducing overfitting like L1,L2, but this is just speculation. However I think the official description should be verified again. Maybe I should open a issue in tensorflow github repository.

@burui11087
Copy link
Author

Post-training quantization is a conversion technique that can reduce model size while also improving CPU and hardware accelerator latency, with little degradation in model accuracy.
From https://www.tensorflow.org/lite/performance/post_training_quantization

I think that's my fault after reread https://tf.wiki/zh/deployment/lite.html#quantization . I find what you talk about is post-training quantization, it will cause accuracy loss. I use quantization-aware training in my daily work, that will make the accuracy better or worse.

@burui11087
Copy link
Author

@burui11087 Could you tell me on which condition the quantized size will be bigger than the original one?

not quantized size, it's accuracy.

@dpinthinker
Copy link
Collaborator

@burui11087
Yes, I just try to talk about post training quantization here.

Thank you for your careful reviewing. I am a beginner in tflite. If you have found any issue in this chapter, or have any idea, please let me know.

@dpinthinker
Copy link
Collaborator

@snowkylin maybe we need add training-aware-quantization content later.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants