Key error for Reshape when trying to convert ONNX model

[samedii]

Do you think this error occuring because the weights name is missing in the ONNX file or because the operation isn't found?

I'm getting the right results if I load and run the model with ONNX runtime at least.

  File "/home/richard/miniconda3/envs/3.9.13/lib/python3.9/site-packages/onnx2torch/converter.py", line 109, in convert
    torch_module, onnx_mapping = converter(onnx_node, onnx_graph)
  File "/home/richard/miniconda3/envs/3.9.13/lib/python3.9/site-packages/onnx2torch/node_converters/gemm.py", line 65, in _
    weights = graph.initializers[weights_value_name]
KeyError: 'Reshape__2177:0'

Here is a link to the ONNX file if you want to try it yourself
https://drive.google.com/file/d/1P_Bl7n2hbUoOhfMh_9UJfgK_N8kP_xjB/view?usp=sharing

I'm trying to make this model https://paperswithcode.com/paper/lit-zero-shot-transfer-with-locked-image-text easier to use for people

[senysenyseny16]

Hi, Richard!

Do you think this error occuring because the weights name is missing in the ONNX file or because the operation isn't found?

The problem is that our implementation is based on the assumption that B (Y = alpha * A' * B' + beta * C) is in initializers, while your LiB-B16B network has GEMM operation where B is tensor, not initializer.

We'll fix it soon.

[samedii]

Okay I tried to debug the code and I think I maybe understand the issue. This might be due to how JAX works (parameters are sent to functional versions of everything)

Do you have a pattern for solving this type of issue already? I can have a go if you can point me in the right direction

[senysenyseny16]

Your model corresponds to ONNX specification, the problem is on our side.

Do you have a pattern for solving this type of issue already? I can have a go if you can point me in the right direction

Bug has already been fixed, the fix will be published tomorrow.

[senysenyseny16]

Richard,

fix has been published on PyPI and conda-forge.

Thanks for finding this error, feel free to open new issues.

[samedii]

Thank you!