New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Quantize nearest_interp and nearest_interp_v2 #38622
Conversation
Thanks for your contribution! |
std::unordered_set<std::string>( | ||
{"concat", "conv2d", "depthwise_conv2d", "elementwise_add", "fc", | ||
"matmul", "nearest_interp", "nearest_interp_v2", "pool2d", | ||
"prior_box", "reshape2", "transpose2", "fusion_gru", "fusion_lstm", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just for my curiosity, what is prior_box
? Do we support that operation?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have found it already, I think that it is the only op that doesn't end with "mkldnn_op.cc"
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, you are right, we are using a native version here which supports all data types including int8.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
std::unordered_set<std::string>( | ||
{"concat", "conv2d", "depthwise_conv2d", "elementwise_add", "fc", | ||
"matmul", "nearest_interp", "nearest_interp_v2", "pool2d", | ||
"prior_box", "reshape2", "transpose2", "fusion_gru", "fusion_lstm", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have found it already, I think that it is the only op that doesn't end with "mkldnn_op.cc"
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM.
This PR supported Nearest_interp int8 and improved "faster_rcnn" int8 model by 2%, ocr_det int8 model by 11%, Retinanet int8 model by 2%
@baoachun Can I merge this PR? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
@Aganlengzi Hi Baoachun approved. Please merge this PR, Thanks ! |
PR types
Performance optimization
PR changes
OPs
Describe
This PR adds:
cpu_quantize_placement_pass.cc
added checking if the list of operators entered by the user contains all the operators supported by quantizationNearest_interp quantization improved the performance by 2% in the "faster_rcnn" model and almost 11% in the "ocr_det" model.