New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

stereo mode #41

Open
wezzoid opened this Issue Mar 27, 2018 · 15 comments

Comments

4 participants
@wezzoid

wezzoid commented Mar 27, 2018

Hi, how do I use this with the Compute Module to publish nodes for 2 cameras?

@realizator

This comment has been minimized.

realizator commented Oct 17, 2018

+1, the same question.
Raspbian supports stereoscopic mode out of the box (https://www.raspberrypi.org/forums/viewtopic.php?f=43&t=85012&sid=ec557617fabfa9942e5baeec16bf862c)

Raspberry Pi Compute module supports 2 cameras, and using a Pi devboard or ready-to-use boards like StereoPi you have native stereoscopic image.
I can help to develper by providing hardware for tests (StereoPi engineering version). @rohbotics, please contact me if you need hardware set for tests and stereoscopic implementation to your driver.

@rohbotics

This comment has been minimized.

Member

rohbotics commented Oct 17, 2018

We use the same firmware and kernel as raspbian, so that side of stereoscopic processing should work.

Can you use our image (https://downloads.ubiquityrobotics.com/pi.html) , and see if you can get PiCamera to output stereo?

@realizator

This comment has been minimized.

realizator commented Oct 17, 2018

@rohbotics, it just works! :-) The only thing I did is put dt-blob.bin (device tree for StereoPi) in a boot partition to enable two cameras support.
Thank you!

scrnsht

@realizator

This comment has been minimized.

realizator commented Oct 17, 2018

@rohbotics, concerning @wezzoid question - how do you think, what is the simplest way to publish nodes for stereo processing?

@rohbotics

This comment has been minimized.

Member

rohbotics commented Oct 17, 2018

I'll have to look into the mmal format for stereo, but the way to do it is basically publish 2 separate images and use stereo_image_proc for combining them into a stereo_msgs/DisparityImage in ROS.

A couple concerns:

  • I think we will be limited to raw publishing, or will have to find a way to cut a jpeg in half to publish it separately
  • Calibration, how do we save calibration data for each camera separately?

Also PRs are welcome :)

@realizator

This comment has been minimized.

realizator commented Oct 17, 2018

As an idea - you can use two cameras separately. In raspivid I use -cs 0 and -cs 1 to use cameras independently and simultaneously (in my case it is front view and rear view on a drone, not stereoscopic mode). The only disadvantage is not so good image synchronization in comparison with -3d single image capture mode.

But as first step for tests it may be Ok.
Is it possible to load and use two instances of your driver and two cameras simultaneously? Is this solve problem with separate calibration data for each camera?

@rohbotics

This comment has been minimized.

Member

rohbotics commented Oct 17, 2018

Being able to do select a camera for each instance of the node would be a really good first step in supporting stereo.

I'll look into it.

@davecrawley

This comment has been minimized.

Contributor

davecrawley commented Oct 17, 2018

@wezzoid

This comment has been minimized.

wezzoid commented Oct 19, 2018

I used the picamera Python lib to get stereo working for my project (https://hackaday.io/project/159212-augmented-mekamon-robot) I put the camera in stereo mode, loaded the data in as an array, chopped it in half and published the left & right frames to ROS, basically. Doing it in Python is slow. To get around this I put the camera in top-bottom stereo mode, which means chopping the image data in half is just one 'cut' instead of one cut per row.

Here's my (amateurish) code: https://pastebin.com/RQ4DMBJj

@davecrawley the normal Pi 3 doesn't have the right pins exposed. Only the compute module allows this. Why Raspberry don't want to add a second CSI connector to their revised boards I don't know.

@realizator

This comment has been minimized.

realizator commented Oct 21, 2018

@wezzoid thank you for your sample code! It is right this example that we were looking for! I will try to combine it with @rohbotics camera selection support from his last pull request after tests.

@realizator

This comment has been minimized.

realizator commented Nov 20, 2018

@wezzoid thank you for your code sharing. After minor changes it works fine:

screenshot from 2018-11-20 10-31-23

BTW I had tried to contact you on hackaday.io but did not received any answer. Are you still there?

@rohbotics

This comment has been minimized.

Member

rohbotics commented Nov 21, 2018

@realizator That is very cool! Looks like I am going to have to pick up a StereoPi 😄

@realizator

This comment has been minimized.

realizator commented Nov 21, 2018

@rohbotics StereoPi will be available after our coming-soon crowdfunding (here is pre-launch page), but I can send you one of my current revision board for experiments right now. I just need your address for parcel.
I wrote you email to send2rohan@... but did not receive a response (possibly my message was marked as a spam). You can write me to realizator[at]gmail[dot]com
I wonder if it is possible to make stereoscopic driver for ROS like other stereoscopic cameras have. It will be compatible with original Raspberry Pi CM3 devboard and with StereoPi. Also we plan to do StereoPi schematic open source, and I hope some companies like Adafruit will help us to make this board mass-producted and available for DIY enthusiasts.

@wezzoid

This comment has been minimized.

wezzoid commented Nov 21, 2018

@realizator - awesome stuff!
And that StereoPi board looks VERY interesting!
I clearly haven't been checking my Hackaday, oops.

@realizator

This comment has been minimized.

realizator commented Nov 26, 2018

@wezzoid, @rohbotics - I put all my files used for tests here: https://github.com/realizator/StereoPi-ROS-depth-map-test
May be they will be useful.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment