IMU input support is needed for point cloud creation

[gabor-meszaros]

IMU data should be considered when you generate the point cloud from velodyne packates.

Whith the current implementation when you see the generated pointcloud frame by frame you can observe duplicated objects:

I think the reason of this artifact is the following:

Let's assume the scan starts in t0 and the red line is the first data in the scan. It detects the big red dot. The scanline moves in counterclockwise direction (red arrow). During the scan the car/lidar moves on the gray arrow. In t1 -before the finishing the revolution, so it is still one frame- the green scan line (started from the t1 point) detects the same object that it detected in t0.

As I read the LiDAR rotates in clockwise direction but I do not want to redraw the figure. The result is the same.

If the car moves with 36 Km/h and the LiDAR rotates on 10hz that means the following:
22.3 mph = 36 km/h = 10m/s; 0.1 sec = 1 m ==> The difference can be 1 meter (3.28 feet). If the car moves faster it even worse.

I think an IMU can help to make it correct (or at least make it better) but we cannot use it on the pointcloud. We have to work with the raw packets.

Sometimes it makes the object recognition difficult because if two object moves close to each other the algorithm detects them one object. It can also result problems in frame registration.

Have you encountered with this problem in the past? Is it possible if it has a different reason? Do you consider it a real issue or it can be treated as "noise".

[jack-oquin]

What frame of reference are you using for the point cloud? Which node or nodelet is publishing the data?

If you are using the cloud node or nodelet, all the data are probably presented in the /velodyne frame, which rotates in the example given. When the device is moving, perhaps with a yaw component, that becomes a problem. It is the reason for using the transform node or nodelet, which converts each packet into the /odom frame using the latest available Odometry tf information. The packets typically arrive more frequently than the odometry updates, but if your Odometry is available at a fairly high rate, it should eliminate the kinds of artifacts you show here.

There are sometimes start-up problems getting the transform in sync with the Odometry, resulting in many error messages until the data are available in a timely fashion. If that is the kind of problem you are seeing, there may be improvements we can make. There is an open issue for it:: #2.

[gabor-meszaros]

Thanks for the reply and sorry for the late reply. I used the cloude nodelet and the frame of reference was the /velodyne frame.

The transform nodelet is a good point, it can help in this. But in this case how to keep it in sync with the packets? I mean during the processing, not only at the start.

Let us see this example:
You use a Velodyne HDL-64E S2.1 LiDAR with the GPS provided by velodyne (1200 rpm/20 Hz). In this case for each packet we have a very precise time information (GPS time). Together with the LiDAR you also use an IMU (but it is a separated device, no connection to the LiDAR) that logs its data with a GPS time stamp into a file. In this case, if everything goes well, you can sync the data later using the GPS time.

How can you do this in ROS?

I can publish the points with the velodyne driver's transform nodelet and I can publish the IMU data as a transformation data, but the sync looks a bit problematic for me. Not only at the start but for a whole replay. I cannot start them exactly in the same time.

Do I misunderstand something? How can we sync the two data without the usage of GPS time stamp (or without any reference)?

[jack-oquin]

I don't have a complete answer to your questions, but I do understand the main issues. Perhaps raising this on http://answers.ros.org may elicit more complete responses from others. Or, it could be discussed on the ros-sig-drivers mailing list.

Basically, the ROS tf package will do the right thing, as long as the times are represented in ROS time format.

So, the problem is converting GPS times to ROS (system) time. The ROS solution for this conversion is the sensor_msgs/TimeReference message. GPS drivers should publish their data in ROS system time, plus a TimeReference message mapping to GPS time.

Your IMU driver needs to understand these issues and publish data with the correct timestamp.

The Velodyne driver currently uses system time estimate of the each packet received. We could make that more precise in two ways: (1) support a ~time_offset parameter as specified in REP-0138, or (2) somehow convert the GPS time supplied by some Velodyne devices to ROS system time using something like a Phase Locked Loop.

I am open to adding either or both of those to the Velodyne driver: (1) is straightforward, while (2) is tricky and might require some assistance.

[gabor-meszaros]

To be honest, I cannot decide in this question. Maybe the ROS system based solution sounds better because -as far as I understand- the synchronization will be more controlled.

I have to dive into the GPS time to ROS time conversion (with TimeReference) because it is new for me. I could not find any tutorial about TimeReference message usage. Where should I publish this message or how can I map the two time?

By the way, the GPS timer that we use for the LiDAR is the standard Velodyne supplied solution. I think it would be useful for others also if the velodyne_driver can handle it somehow. I do not know how much do you know about the Velodyne's GPS timer solution, but in summary it works like the following.

Without any references it is impossible to sync the LiDAR data with other sensor data. To make synchronization possible Velodyne provides a device that has a special GPS module. If you connect this device to the LiDAR then it replaces the last 6 bytes of each packet to 4 bytes GPS timestamp and 2 other bytes which is not that important for us now (so the size of the packet is the same). The 4 bytes of the GPS timestamp represents the microseconds from the top of the hour.

You can read more on the pages 14 and 26 here.

Of course it in the case of HDL-64E S2.1. I checked the manuals and the situation may a bit trickier. It seems like you cannot connect this additional GPS device to a HDL-64E (so the meaning of the packet's last 6 bytes is different). But a HDL-32E always have this GPS module connected. I am not sure about the last two models because we only have a HDL-64E S2.1.

What do you recommend?

[jack-oquin]

Let's discuss this on the ROS drivers SIG mailing list. Several people there understand these issues better than we do, and the subject is relevant for other drivers.

@ericperko has a new nmea_navsat_driver which accepts NMEA sentences, defined by a nmea_msgs/Sentence message. This is explicitly intended to support devices like the Velodyne. You may find those API Reviews interesting, especially the most recent one.

Mapping GPS time to system time is tricky, but important. Essentially, they are two separate time sources and will diverge. The usual solution for a problem like that is a Phase-locked loop. In this case, we need a software implementation. While I understand the concept, I probably need help implementing it in the driver. Since it is a common problem for many drivers, I would favor implementing it in a separate package with a supported API that the Velodyne driver and others can use. Perhaps other driver developers would be willing to assist.

For the 32E, GPS information is provided to a different UDP port, so it could be handled completely by a separate driver. Since my device has no GPS, I have favored a separate implementation, so I don't have to maintain it.

However, the 64E S2.1 does need direct support in the driver. We may also decide to add a parameter for using converted GPS time for each packet. If we do that, it should also handle the 32E GPS packets. I am open to providing that. Others will find it useful.

[gabor-meszaros]

For the 32E, GPS information is provided to a different UDP port, so it could be handled completely by a separate driver.

Yes, I agree. But the GPS timestamp (not the position) is part of the data packet and its conversion to ROS time should be part of the driver. The GPS timestamp represents the microseconds from the top of the hour, so we have to collect the year, month, day, hour information from the previous packages too. Or somehow we have to handle it at the conversion.

@ericperko has a new nmea_navsat_driver which accepts NMEA sentences, defined by a nmea_msgs/Sentence message. This is explicitly intended to support devices like the Velodyne. You may find those API Reviews interesting, especially the most recent one.

I read it and it will be a good reference for me when I implement the publisher of the IMU information with the GPS -> ROS time conversion.

I am going to write to the ROS drivers SIG mailing list soon and I am going to link this topic too.

[jack-oquin]

I agree that the time synchronization needs to be done in the driver, because the data time stamps need to match the position, orientation and transform messages. In many cases, no one will need the sensor_msgs/TimeReference message, though we should publish it for anyone who wants it.

The main difficulty is clock synchronization, not just the format conversion. The system clock and the GPS time source will tick at slightly different rates, and we must compensate when mapping one to the other.

This looks like a fun project. I'll join the discussion on ros-sig-drivers and also inquire on answers.ros.org whether anyone already has a PLL package with an appropriate open license.