Onnx/Torch model transpilation / Parsing to MLX. (And Supporting onnx weights)

[artificalaudio]

Everything I do is on the new Mac chips, so I'm very excited. I use C++/cmake/Juce and onnx/torch script. I'm trying to squeeze every last second of performance to do real time inference locally. I've tried swapping backend in onnx for CoreML, but didn't notice an improvement.

In short, I've tried out a few ways to increase inference speed using onnx and torch, I'm very intrigued by mlx.

This is this question. Is there any plan to support parsing of a graph, such as from Onnx? This would be an incredibly useful feature.

I've had a brief look at export weights, and looks like you're parsing and renaming some layer names, and exporting to a format that can be loaded in mlx. That script is for torch weights, how about one for onnx files?

I think it would be amazing if you supported parsing of graphs, I appreciate there's an example with 200 lines of code, the llama inference code. I think it could be helpful (potentially) if you were to show the original for comparison, if there's a torch equivalent, show the major conceptual changes/differences in setting up a model.

(You probably think I'm an idiot) The reason for showing differences is that; if there is a difference you highlight them, like remove torch calls, swap xname for yname, then we can visualise the changes needed and potentially make a manual model parser.

And please forgive my naivety, is there anything that would prevent a model parser currently?

For context I've built very simple neural net libraries for MLPs in C++ without any matrix libraries Eigen etc, and have a persistent obsession to understand the lower levels. I would be happy to invest some of my time to make an onnx parser if you could confirm the framework is feature rich enough. Or, should the first tool be a model checker, to see if a model can be converted, whether the ops are supported?

[rosaboyle]

+1

[awni]

Closing as dup of. #290