Modify RunONNXModel.py for pip install

[chentong319]

The goal is to create a python package so that we can run an inference like onnxruntime in python script. Since the RunONNXModel.py provides most of the necessary functionality, I added extra support in that script. I tested with a local pip install. We can upload the package later.

Major changes:

1. The code for package, onnxmlir, is added in utils.
2. The main code is modified from utils/RunONNXModel.py. Therefore, we can run inference in onnxruntime way, or our RunONNXModel.py way.
3. Related document is added in docs.
4. Details of the package can be found at utils/onnxmlir/README.md, and example can be found in utils/onnxmlir/tests

Example:

import numpy as np
import onnxmlirrun

a = np.zeros((3,4,5), dtype=np.float32)
b = a + 4
r = onnxmlirrun.RunONNXModel.onnxmlirrun(compiled_so="/Users/chentong/Projects/onnx-mlir/build/test_add.so", inputs=[a,b])
print(r)
r = onnxmlirrun.RunONNXModel.onnxmlirrun(onnx_model="/Users/chentong/Projects/onnx-mlir/build/test_add.onnx", inputs=[a,b])
print(r)

We may have different name for the functions and the package. Comments are welcome.

[AlexandreEichenberger]

@chentong319 Can you add a reference to the ORT interface that this PR is imitating.

In your first example

r = onnxmlirrun.RunONNXModel.onnxmlirrun(compiled_so="/Users/chentong/Projects/onnx-mlir/build/test_add.so", inputs=[a,b])

was the onnx_model parameter omitted on purpose? Just for me to understand better the interface.

[chentong319]

@chentong319 Can you add a reference to the ORT interface that this PR is imitating.

In your first example

r = onnxmlirrun.RunONNXModel.onnxmlirrun(compiled_so="/Users/chentong/Projects/onnx-mlir/build/test_add.so", inputs=[a,b])

was the onnx_model parameter omitted on purpose? Just for me to understand better the interface.

Yes, onnx_model is not needed if the compiled .so is provided. By the way, the two onnxmilrrun.RunONNXModel.onnxmlirrun commands are two independent ways to do inference.

[AlexandreEichenberger]

Got it, so we can provide a onnx file or a pre-compiled binary, smart, thanks.

[tungld]

Should we check if both onnx_model and compiled_so are not given?

[AlexandreEichenberger]

Make sense, and maybe if none are given, an error?

[chentong319]

I added the check whether both onnx_mdel and compiled_so are given or not give (XOR).

[tungld]

onnxmlirrun.RunONNXModel.onnxmlirrun

My two cents here. For the package name (the first onnxmlirrun), just onnxmlir is enough to avoid typing two Rs.
I would remove RunONNXModel, but it seems not straightforward unless we reorganize RunONNXModel.py.

Anyway, we have several python utilities, perhaps it's time to reorganize them into a single python package.

[AlexandreEichenberger]

Version number here is different than onnx-mlir/VERSION_NUMBER. Is that on purpose?
Also the number here is different than the version listed in the file below.

[chentong319]

Fixed. I used a version number different from the onnx-mlir. The version for onnxmlir is for upstream onnxmlir to pip repo. It could be viewed as a pre-required package.

[AlexandreEichenberger]

May I suggest a 0.1.0 and move to a 1.0.0 once we have the installation of the compiler with it at least one z?

[AlexandreEichenberger]

Is this an identical copy? If so, at a minimum there should be a comment stating that they should stay in sync. Or maybe we could have a single version and we could "build" the pip structure by copying files in the right place. That would be my preference.

[chentong319]

Added a target in CMake to check the files.

[AlexandreEichenberger]

Check error message if file does not exist.

[AlexandreEichenberger]

TODO: let's try to have parallel default and/or as set in onnxruntime.

[AlexandreEichenberger]

TODO: sync also with #2899 with respect to catching errors/crasshes

[chentong319]

Check error message if file does not exist.

onnx-mlir will report the error if file does not exist. Leave this to the TODO list to catch all the errors gracefully.

[AlexandreEichenberger]

May I suggest a 0.1.0 and move to a 1.0.0 once we have the installation of the compiler with it at least one z?

[AlexandreEichenberger]

Thanks for having it in sync.

[chentong319]

@jenkins-droid test it!

[hamptonm1]

@jenkins-droid test this please

[AlexandreEichenberger]

LGTM, thanks for the updates

[jenkins-droid]

Jenkins Linux ppc64le Build #14467 [push] Modify RunONNXModel.py f... started at 11:56

[jenkins-droid]

Jenkins Linux amd64 Build #15437 [push] Modify RunONNXModel.py f... started at 10:46

[jenkins-droid]

Jenkins Linux s390x Build #15444 [push] Modify RunONNXModel.py f... started at 11:46

[jenkins-droid]

Jenkins Linux amd64 Build #15437 [push] Modify RunONNXModel.py f... passed after 1 hr 7 min

[jenkins-droid]

Jenkins Linux s390x Build #15444 [push] Modify RunONNXModel.py f... passed after 1 hr 29 min

[jenkins-droid]

Jenkins Linux ppc64le Build #14467 [push] Modify RunONNXModel.py f... passed after 2 hr 1 min