Does NNstream support Jetson board such as Jetson AGX Orin, and ML framwork TensorRT?

[liuhao-97]

Does NNstream support Jetson board such as Jetson AGX Orin, and ML framwork TensorRT?
Besides, is it possible to use fit nnstream into NVIDIA deepstream?

Thanks!

[taos-ci]

cibot: Thank you for posting issue #4459. The person in charge will reply soon.

[myungjoo]

1. Yes, TensorRT is supported.
2. Yes, as long as you implement a decoder/converter between deepstream streams and nnstreamer streams.
   • Deepstream has "tensors" as metadata of media streams; not considering tensors as native data types of gstreamer.
   • NNStreamer has "tensors" as yet another media streams; considering tensors as native data types.

[liuhao-97]

Thanks for answering. @myungjoo

BTW does NNStreamer accept Udp package with raw tensor as udp input? I am asking because I am doing model splitting, which the model is splitted into two parts and these two parts are executing on two devices, and the intermediate feature map are sent from one device to the other. I am wondering if NNstreamer support this feature.

Thanks!

[myungjoo]

BTW does NNStreamer accept Udp package with raw tensor as udp input?

Yes. Actually, that's what GStreamer's base plugins do, udpsrc and udpsink, if what you want is a raw & basic connectivity. https://gstreamer.freedesktop.org/documentation/udp/udpsrc.html?gi-language=c

For example. if you are accepting UDP packets of tensors:

udpsrc port=10000 ! other/tensors,format=static,dims=.... ! tensor_filter .... ! ....

If you want something more than that, you may use edgesrc/edgesink with TCP or MQTT/Hybrid.