Available INT4 ONNX models?

[laumecha]

Hello everyone,

Does anyone know where I can find INT4 ONNX quantized models with reasonable accuracy?

[ramkrishna2910]

Hi @laumecha
Can you add some more details on what models you are specifically looking for?
You can find some int4 onnx models in the Huggingface here.

[laumecha]

Thank you very much for responding. I was looking for classification or object detection segmentation models.

[ramkrishna2910]

We currently dont have any int4 models available. But you can use publicly available tools like the neural-compressor to generate int4 models.

[ramkrishna2910]

Closing due to inactivity.