Point Cloud Data Not Visible On Calibration Toolkit

[Eric-Gonzalez]

Point cloud data is not displayed using the Camera -> Velodyne calibration type.

Steps to Reproduce

1. Playback bag containing /camera/image_raw, /camera/camera_info, and /points_raw topics.
2. Launch Calibration Toolkit from the Sensing tab of Autoware.
3. Select the image topic of /camera/image_raw.
4. Select Calibration Type of Camera -> Velodyne
5. Center calibration board in front of camera
6. Click "Grab"

Expected Behavior

We expect to see an image similar to "CalibrationToolkit_Manual.pdf" where the right image contains the Velodyne point cloud. Instead the image in both the top right and bottom right remain black.

Debug Information

Confirm that topics are emitting messages

subscribed to [/points_raw]
average rate: 9.538
    min: 0.104s max: 0.106s std dev: 0.00060s window: 9

subscribed to [/camera/image_raw]
average rate: 20.031
    min: 0.048s max: 0.053s std dev: 0.00156s window: 19

binning_x: 1
binning_y: 1
roi: 
  x_offset: 640
  y_offset: 360
  height: 480
  width: 640
  do_rectify: True

---
header: 
  seq: 13134
  stamp: 
    secs: 1478123855
    nsecs: 554909520
  frame_id: camera
height: 480
width: 640
distortion_model: plumb_bob
D: [-0.04174, -0.890952, 0.014123, -0.007406, 0.0]
K: [2152.445406, 0.0, 268.01097, 0.0, 2166.161453, 302.594211, 0.0, 0.0, 1.0]
R: [1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0]
P: [2147.012207, 0.0, 266.243042, 0.0, 0.0, 2157.508545, 303.462138, 0.0, 0.0, 0.0, 1.0, 0.0]

Screenshot

[alexanderhmw]

Got it. I will work on it to find the bug~
Thanks for the issue report!

[alexanderhmw]

Hi Eric,

First I have a question about the Velodyne timestamp. From the Github issue image, I can confirm that the calibration toolkit received the "/points_raw" message because the top right window's tab displayed the timestamp "21:58:28:659". If it didn't receive the message, the tab would only show "timestamp". However, the thing I cannot understand is the huge gap between the camera's timestamp (22:23:48:574) and the Velodyne's timestamp (21:58:28:659). So first please check the synchronization status between these two message streams, and play them with "--clock" option as well as setting the "use_sim_time" to be true. The timestamp problem is the first thing we need to handle.

Then, two possible problems come to my mind, and both of them are in the refreshVelodyne function:
https://github.com/CPFL/Autoware/blob/master/ros/src/sensing/fusion/packages/calibration_camera_lidar/CalibrationToolkit/calibrationtoolkit.cpp#L698

(1) The color of the Velodyne points may be black (line 704 - 712)
Click the top right window, and then press "b" to change the background color. If you still cannot see the points, please go to (2). If you can see the points, your Velodyne data field is different from ours. We use the fourth float value (intensity) to render the points. If it is always zero (not used), it will be black in this application. So please check the field name of your Velodyne data following this link: http://wiki.ros.org/pcl/Overview#Common_PointCloud2_field_names

(2) The GLViewer derived from the QGLWidget doesn't work? <1% possiblility~ (line 714 - 730)
I don't know what happens to the Qt's GLWidget. But if the Velodyne data is correctly loaded, the only problem is the visualization conducted by OpenGL. Please check the OpenGL version >=2.0, and test the GLViewer library (https://github.com/CPFL/Autoware/tree/master/ros/src/util/packages/RobotSDK/glviewer) in a simple test application, and also test the "glDrawArrays" function.

I think the first problem is the possible one. Please output the variables "n"(number of the points to be displayed, maybe zero, then the height and width of PointCloud2 are different with ours), "colors" or "base".

Best,
HMW-Alexander

[alexanderhmw]

Hi Eric,

Another problem is the chessboard's pattern number (number of inner corners: http://docs.opencv.org/2.4/modules/calib3d/doc/camera_calibration_and_3d_reconstruction.html#findchessboardcorners)
According to your chessboard, the pattern number is 6x8, not 7x9.

Best,
HMW-Alexander

[Eric-Gonzalez]

Hey Alexander,

The blank screen was a combination of two issues.

1. You were correct; the points were not visible because they were rendered black against a black background.
2. The default GLViewer perspective was oriented away from the points. Using a combination of the WASD, and QE keys I was able to bring the points into view.

I'll continue the calibration process when I return to the vehicle.

🙌 Thank you for the assistance!

[alexanderhmw]

My pleasure!

[wsung1]

Dear @Eric-Gonzalez, @alexanderhmw
While sensor calibrating, I have point cloud data "visible" through RViz

but "not visible" on calibration toolkit.

As shown above, I changed the background color but still I couldn't see any points. Thus, I think I have to work on GLViewer but only from your descriptions I can't figure out how to do exactly.
e.g.) The default GLViewer perspective was oriented away from the points. Using a combination of the WASD, and QE keys I was able to bring the points into view.
If you don't mind, would you please elaborate it for me?

[yulan0215]

Hey Alexander,

The blank screen was a combination of two issues.

1. You were correct; the points were not visible because they were rendered black against a black background.
2. The default GLViewer perspective was oriented away from the points. Using a combination of the WASD, and QE keys I was able to bring the points into view.

I'll continue the calibration process when I return to the vehicle.

🙌 Thank you for the assistance!

Hello,I want to ask how did you solve the problem of synchronization between camera and lidar? I was stuck in this step, thank you very much.

[JWhitleyWork]

@yulan0215 Per our support guidelines, please create a post on ROS Answers instead of continuing this (very old) conversation here.