Incompatible dimensions for matrix multiplication Error in StarNet model when doing InferenceSession

[rafaelagrc]

Hello.
I have a Text Recognition model (StarNet - https://github.com/clovaai/deep-text-recognition-benchmark) which I am trying to dynamically quantize to reduce its size and deploy it on mobile.

When I export the model from PyTorch to ONNX, I can successfully run and do inference.
However when I take the ONNX model, apply the "quantize_dynamic" function and start inference, I get the following error:

Fail Traceback (most recent call last)

File ~/anaconda3/envs/text_recog/lib/python3.8/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py:335, in InferenceSession.init(self, path_or_bytes, sess_options, providers, provider_options, **kwargs)
332 disabled_optimizers = kwargs['disabled_optimizers'] if 'disabled_optimizers' in kwargs else None
334 try:
--> 335 self._create_inference_session(providers, provider_options, disabled_optimizers)
336 except ValueError:
337 if self._enable_fallback:

File ~/anaconda3/envs/text_recog/lib/python3.8/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py:368, in InferenceSession._create_inference_session(self, providers, provider_options, disabled_optimizers)
366 session_options = self._sess_options if self._sess_options else C.get_default_session_options()
367 if self._model_path:
--> 368 sess = C.InferenceSession(session_options, self._model_path, True, self._read_config_from_model)
369 else:
370 sess = C.InferenceSession(session_options, self._model_bytes, False, self._read_config_from_model)

Fail: [ONNXRuntimeError] : 1 : FAIL : Load model from /xSTAR-Net_artifacts/Onnx_models/finetuned_by_Rafaela_dyn_quant.onnx failed:Node (Gemm_334_MatMul_quant) Op (MatMulInteger) [ShapeInferenceError] Incompatible dimensions for matrix multiplication

[yufenglee]

@rafaelagrc, could you please share the model with us to repro the issue?