ONNX Runtime

[rismanadnan]

We are aware that NNStreamer supports the most popular inference engines, such as TF Lite, TVM (experimental), and DeepViewRT engine. Is there any plan to support ONNX model files and runtime? Example from gstonnx: support neural network inference on video via ONNX (onnxobjectdetector, onnxsuperresolution) in this link: https://www.youtube.com/watch?v=rpFRS5nKxqI

[myungjoo]

We do not have plans to support ONNX runtime. It can be added as a 6-weeks practice project for a newly hired (On-the-Job Training) if we have a new one. Actually, TVM was a practice project for a newly hired and MXNet was an undergrad term project at a university.

Anyway, the gstonnx you've mentioned is not proper for actual applications and services. It accepts video and outputs video, which is for simple demonstration. In actual services, it should accept arbitrary data (text, audio, video, some sensors, some inference results of other neural networks, ...) and provide essential data for application threads or other neural networks.

[myungjoo]

Anyway, if we happen to have an intern this winter, we can assign ONNX runtime subplugin as his/her internship project. In general, it is supposed to be a C++ subclass of "tensor_filter_subplugin" abstract class, with less than 1000 lines; writing a tensor_filter subplugin as a subclass is the recommended method. The most complex tensor_filter subplugin is a little shorter than 2000 lines.

[rismanadnan]

Thanks for reply! We will consider to put engineer and interns on ONNX RT support. I agreed with you with gstonnx limitation. We will take look deeper on tensor_filter subplugin development guide, especially on how to develop shared library and loaded by tensor_filter. We will ask questions in this thread if we decide to build this subplugins. We also have plan to support GLOW in future once we are able to write tensor_filter subplugin.