Garbled image problem on usb_cam

[rawk-v]

@bosch-ros-pkg
I am testing usb_cam_node with a usb camera, when I use the follow launch file, everything is ok,

<launch>
    <node name="camera" pkg="usb_cam" type="usb_cam_node" output="screen">
        <param name="video_device" value="/dev/video0" />
        <param name="image_width" value="640" />
        <param name="image_height" value="480" />
        <param name="pixel_format" value="yuyv" />
        <param name="framerate" value="30" />
        <param name="camera_frame_id" value="usb_cam" />
        <param name="io_method" value="mmap" />
        <param name="camera_info_url" type="string" value="file://$(find pixhawk)/launch/camera.yaml" />
    </node>

    <node name="viewer" pkg="image_view" type="image_view" output="screen">
        <remap from="image" to="/camera/image_raw" />
    </node>
</launch>

but when I change pixel_format to mjpeg, the video become garbled

v4l2-ctl --list-format-ext -d /dev/video0 output:

ioctl: VIDIOC_ENUM_FMT
    Index       : 0
    Type        : Video Capture
    Pixel Format: 'MJPG' (compressed)
    Name        : MJPEG
        Size: Discrete 1920x1080
            Interval: Discrete 0.033s (30.000 fps)
        Size: Discrete 1280x720
            Interval: Discrete 0.017s (60.000 fps)
        Size: Discrete 1024x768
            Interval: Discrete 0.033s (30.000 fps)
        Size: Discrete 640x480
            Interval: Discrete 0.008s (120.101 fps)
        Size: Discrete 800x600
            Interval: Discrete 0.017s (60.000 fps)
        Size: Discrete 1280x1024
            Interval: Discrete 0.033s (30.000 fps)
        Size: Discrete 320x240
            Interval: Discrete 0.008s (120.101 fps)

    Index       : 1
    Type        : Video Capture
    Pixel Format: 'YUYV'
    Name        : YUV 4:2:2 (YUYV)
        Size: Discrete 1920x1080
            Interval: Discrete 0.167s (6.000 fps)
        Size: Discrete 1280x720
            Interval: Discrete 0.111s (9.000 fps)
        Size: Discrete 1024x768
            Interval: Discrete 0.167s (6.000 fps)
        Size: Discrete 640x480
            Interval: Discrete 0.033s (30.000 fps)
        Size: Discrete 800x600
            Interval: Discrete 0.050s (20.000 fps)
        Size: Discrete 1280x1024
            Interval: Discrete 0.167s (6.000 fps)
        Size: Discrete 320x240
            Interval: Discrete 0.033s (30.000 fps)

And the error message:

[mjpeg @ 0x223e460] overread 8
[mjpeg @ 0x223e460] error count: 68
[mjpeg @ 0x223e460] error y=38 x=1
[mjpeg @ 0x223e460] error count: 64
[mjpeg @ 0x223e460] error y=34 x=1
[mjpeg @ 0x223e460] error count: 68
[mjpeg @ 0x223e460] error y=40 x=4
[mjpeg @ 0x223e460] error count: 64
[mjpeg @ 0x223e460] error y=42 x=1
[mjpeg @ 0x223e460] error count: 64
[mjpeg @ 0x223e460] error y=31 x=0
[mjpeg @ 0x223e460] error count: 64
[mjpeg @ 0x223e460] error y=30 x=37

My testing enviroment:
Ubuntu 14.04.3
ROS indigo
I've search almost the whole web, can't figure it out, please help! Thanks

[rctoris]

Not sure what might cause this. Is there a reason you need to read in MJPEG instead of YUYV?

[rawk-v]

@rctoris Thanks for your reply.
Because I am working on SVO, so I need a high fps to avoid image blur, but according to v4l2-ctl --list-format-ext -d /dev/video0, my usb camera fps can only reach to 30fps when set pixel_format to YUYV, and I can get 120fps when set pixel_format to MJPEG.

By the way, when I use luvcview in ubuntu, I can get a normal image at 120fps, 640x480 in MJPEG format.

[tobyhijzen]

Might be related to this: http://answers.ros.org/question/217382/garbled-image-problem-on-usb_cam/ I'm also having similar problems.

[LucasMahieu]

hi, i'm having exactly the same problem, did you find any solution ?

[rawk-v]

@LucasMahieu Finally, I wrote a another mjpeg2rgb function with libjpeg in usb_cam.cpp. And it works.

[LucasMahieu]

Thanks a lot @rawk-v for your reply !
We are in school project and don't have enough time to try it,
so using ubuntu 16 solve the problem too.

did you success to get good result with SVO ?
We are also working on it, with a quadricopter

[rawk-v]

I do get some nice desktop graph result with pixhawk IMU and a USB camera, but I don't think it is good enough to test in a real quadcopter with my current camera (a 120fps VGA camera with rolling shutter), as the SVO instruction said, a global shutter with big FOV may get same result as the video. So recently, I bought another camera to test it (I have not yet got it), and I will test it in a quadcopter soon.

[LucasMahieu]

nice, so you are always working on it !
It is for a personal project ?

With camera are you waiting ?
We are not really good in that stuff ... We have this one http://www.elpcctv.com/full-hd-2mp-usb-camera-black-and-white-module-p-231.html , and the result is nos very robust.
We success to track the camera, but the tracking stop very easily

[rawk-v]

Right, not a personal project, just for study.
I am waiting for the new camera. The camera you are using seems very similar to ours.
This is a rolling shutter camera, you can use a PTAM calibration method and ATAN model to generate the calibration params. The result may be better.

[LucasMahieu]

Oh great !
Yes we use the PTAM Calibration and we also tried the PINHOLE model

If you want to see the result of our project, it is in French because of our school but its is complet http://fablab.ensimag.fr/index.php/Robotique_:_implémentation_d%27un_système_d%27odométrie_visuelle_embarqué_sur_un_drone

[rawk-v]

Wow! It is really a complete procedure(Google translate is awesome :) ), maybe sometime I should write down mine too for sharing.
Instead of Naze, I use a pixhawk board, and now I am working on OFFBOARD control with ROS running on Odroid, so I can test it in real copter.

[LucasMahieu]

Yes, we found it is important to keep a written trace of our works.

Yesterday I searched about the pixhawk, it looks to be very adapted and more powerful than naze. Are you student in ETHZ ? It saw pixhawk was developed by ethz's student.

We thought to it too, but how do you want to send the camera stream to the OFF-BOARD computer ? Do you think it will be enough ?

[rawk-v]

Nope, I am a student in china, and they did a great jobs!
I use Odroid XU4 for a companion computer with pixhawk, and Odroid running Ubuntu with ROS middleware, since SVO has a ROS node to publish vision pose data, I use MAVROS to communicate with pixhawk. And then use a msf data fusion framework to combine /svo/pose and /mavros/imu/data together to generate local position.
Also mavros node has subscribe ROS topic like /mavros/setpoint_position/local to send command to pixhawk(PX4 firmware has support for a so-called OFFBOARD mode).
So, I can control pixhawk with OFFBOARD mode according to local position data.

[LucasMahieu]

Really great ! It is a big project, I wish you will get good result !
Good luck

[rawk-v]

Thank you :)