Mediapipe VS Nvidia's Deep Stream

[ShettyHarapanahalli]

Hi,

Just curious to understand the differences between the Mediapipe and Deepstream (https://developer.nvidia.com/deepstream-sdk), are these both addressing the same problem space. Maybe solutions spaces could be different, but are problem spaces same?

Best Regards,
Suman

[mcclanahoochie]

Hi,
Good question; at first glance, Medaipipe (MP) and DeepStream (DS) may seem similar. At the core of it, they both provide a framework that can run an inference task.
There are a few important differences:

• Data types: DS supports only video/image streams, while MP operates on generic packets (which can include video, audio, image, primitives, or any user-defined object types)
• Graphs: Data flow in both frameworks are described by a graph. DS is built on top of Gstreamer, which has a strictly linear pipeline model with very basic control flow. MP graphs offer more flexibility in design, and allows for very complex graph construction, including dynamic control flow and back edges.
• Hardware/OS: DS is restricted to high-end desktop Nvidia GPUs (and appears to be Linux only). MP supports running on desktop/mobile CPUs and mobile GPUs (via OpenGL), and desktop GPU support is planned. MP runs on Android/iOS/Linux/OSX.
• Tools: MP provides a powerful graph visualizer for interactive building and analyzing graphs, including graph performance tracing. I think DS relies on existing Gstreamer visualization tools like graphviz or dot (which is not interactive).
• Plugins/Calculators: Each node in a DS pipeline is a Gstreamer filter (limited to how you write Gstreamer plugins). Each node in a MP graph is a Calculator, representing a very generic operation[s] to any or multiple packet stream types, also not limited to inference or even machine learning at all.

Hope that can help distinguish the two frameworks.