ONNX support

[bezero]

Is there an ONNX model support planned for TRTIS? There are some commits available at #283 with regards to ONNX Backend. If yes, when it is estimated to be available?

[deadeyegoodwin]

Yes, we are currently working on integrating ONNXRuntime into TRTIS to give "native" ONNX support (so you won't need to convert to Caffe2 or TensorRT to run the model). There is a good chance that minimal support will be available in 19.06. Unless there is an unexpected drain on developer time it will certainly be in 19.07.

[rakelkar]

Dumb question - 19.07 of what? Per releases TensorRT server is at 1.2.0... 19.07 seems a way out so am assuming you are referring to something else? :-)

[bezero]

@rakelkar each month there is a new update is released. They are released under the branch with current month tag. 19.07 means year 2019 month Jul (07)s release.

[rakelkar]

Thanks so did this support make it in 19.06?

[bezero]

19.06 will be released closer to the end of the month, after 20th.

[deadeyegoodwin]

ONNXRuntime support is in 19.06 (so will be released later this month as @bezero mentioned). You can find it now on master. We will update the master version of the documentation today or tomorrow and it will contain some information on using ONNXRuntime.