Use it with Intel Realsense

[niknaim]

Very nice work. Thanks a lot. It brings me a lot forward in my project in which I need the object detection. One Question: Is it possible to connect a Realsense Camera to it and get the distances of the obejcts?

[domisjustanumber]

Hey @niknaim thanks for the kind words!

Mediapipe only takes a 2D video image as an input and infers distance purely based on that, so you can't feed it a depth map directly.

That being said, you can probably look up the xy position of each point returned by MediaPipe and then overwrite the Z coordinates with the one you get from the RealSense camera to get more accurate depth positions.

[niknaim]

Thanks for your fast reply. how can I get the the xy position of all points? I only see the xy positions, with, height ans scale of the objects tracked by the object detection. Also I dont see any Z ccordinate, which I could overwrite.

[domisjustanumber]

oh if you're object detection, you only get the xy coordinates of the box containing the object. You could still add Z positions to those values from your RealSense data and/or crop the depth data to the same box and take an average depth of the pixels in the box?

There's lots of options, but they're all beyond the scope of MediaPipe