You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In this repository, we demonstrate CPU quantization only. For the GPU quantization, please visit https://github.com/Microsoft/onnxruntime. Thank you for your interest.
Hi, I followed the example and successfully converted the model to int8 on GPT2 model.
but the model generated with some Customized onnx op:
Such as
DynamicQuantizeLinear
,FastGelu
, how to converted it to tensorrt?the int8 model was 400M compares with original 1.2G. much more small, if can inference via tensorrt, it can be massively accelerated.
The text was updated successfully, but these errors were encountered: