Ideal Sensor Suite for 3D Mapping and Localization

[saimanoj18]

Thanks for this amazing package.

We are planning to employ Google Cartographer for 3D mapping of outdoor large scale environments of few kilometers for self driving car localization and related applications.

We are yet to buy the sensors and from your experience, which sensors and their configuration works best for cartographer ??

1. Which IMUs should we prefer? Could you give us some specifications such as bias or error ?
2. It was said that two velodyne 16's are a good choice and work out of the box. Can someone elaborate on this ? Is one Velodyne 32 is better than two Velo 16's ?
   What kind of configuration should be have ?(One at the front of the vehicle and other at back with tilted angles ?)

3.It is said that we should constrain it in 6 DOF. Could you say, in which way do we miss the aspect of constraining if we use one Velo 16 and how does that get nullified when we use two Velo 16's.

Thank you so much!

[gaschler]

self driving car localization

Don't forget to integrate satellite navigation. (see NavSatFix in cartographer_ros)

Which IMUs should we prefer? Could you give us some specifications such as bias or error?

IMU is not very critical. Make sure you get the actual acceleration and not overly filtered output.

It was said that two velodyne 16's are a good choice and work out of the box. Can someone elaborate on this ? Is one Velodyne 32 is better than two Velo 16's ?

Ideally, scans should cover the entire horizon. So I'd say two 16s (front and back) are better than one 32.

3.It is said that we should constrain it in 6 DOF. Could you say, in which way do we miss the aspect of constraining if we use one Velo 16 and how does that get nullified when we use two Velo 16's.

Sorry, I am not sure what the question is.

Hope this helps.

[cschuet]

One more suggestion: placement of sensors is also critical. To get good localization performance in dense traffic you want to make sure that you see enough static geometry to track the movement of the car in a sea of surrounding cars. We capture nearby geometry in a high resolution hybrid grid and far-away geometry in a low-res hybrid gird. You can tune the weights between the two.

[saimanoj18]

Thank you for your replies.

1. Any low cost IMU's that you have experimented with, which are good for cartographer based mapping and localization applications ? I am thinking of getting XSENS MTI-G 700 series GPS IMU. It has a Gyro bias of 10 degree per hour. https://autonomoustuff.com/product/xsens-mti-g-710/

   Could you please elaborate on the advantages of using NavSatFix topic ?

2. In that case, does mapping happen with Lat and Long coordinates, rather than starting from origin ?

3. I am guessing that the position covariance will be incorporated into the mapping routine. In that case, could you comment on how it would effect the accuracy when the GPS data has inaccuracy of upto 5 meters.

4. Or is it that Cartographer employs GPS data only to find loop closures ?

Thank you

[cschuet]

@wohe @spielawa who contributed GPS support to keep me honest

Any low cost IMU's that you have experimented with, which are good for cartographer based mapping and localization applications ? I am thinking of getting XSENS MTI-G 700 series GPS IMU. It has a Gyro bias of 10 degree per hour. https://autonomoustuff.com/product/xsens-mti-g-710/

I don't have any concrete hardware suggestions. Maybe others here do. I just want to mention that low bias is good but good time synchronization between sensors is also super important. @damonkohler was telling me that the clock line from GPS sensors is often used for time synchronization. Not sure though whether all IMUs support that.

Could you please elaborate on the advantages of using NavSatFix topic ?

In contrast to odometry which describes an incremental, relative change in pose, NavSatFix allows you to specify a constraint to a fixed frame in the world, i.e. an absolute pose, in terms of longtitude an latitude. This would e.g. allow you to upper bound local SLAM drift (if you have good GPS signal).

In that case, does mapping happen with Lat and Long coordinates, rather than starting from origin ?

Right.

I am guessing that the position covariance will be incorporated into the mapping routine. In that case, could you comment on how it would effect the accuracy when the GPS data has inaccuracy of upto 5 meters.

Actually we don't look at the covariance of the position signal. There is a global weight that you can set to define how much Cartographer should trust the GPS signal in loop closures. Filtering out unreliable GPS messages could be performed by a ROS node that subscribes to messages from your GPS sensor and forwards only those that it deems reliable enough. You should still inject an empty GPS message into Cartographer when you have filtered out a real message though to ensure the sensor queues don't stall.

Or is it that Cartographer employs GPS data only to find loop closures ?

Right, GPS is only used in the global SLAM part, i.e. it enters the pose graph as constraints.

[saimanoj18]

Thank you so much @cschuet for those detailed answers. Good day!

[cschuet]

Closing as answered.

[ludennis]

Hi @cschuet,

I am also trying to use the "use_nav_sat" option for cartographer. And I have obtained my gps topic from a rosbag. I understand that I would need to replace every unreliable gps message with an empty gps message in order to not block the queue. However, I don't know what would cartographer consider as an empty gps message, and I am guessing replacing all the fields in NavSatFix with either an empty string ('') or zeros would do the trick, or something else?

Thanks in advance,

Dennis Lu

[wohe]

@ludennis You linked the correct documentation: sensor_msgs/NavSatStatus contains whether this is actually a fix. You have to set status.status to STATUS_NO_FIX.

[ludennis]

Thank you for the reply @wohe! I will give that a try this week and report. Thanks again.

[ludennis]

Hi @wohe,

Sorry for the late respond, but I was able to produce maps with NavSatFix. For bad/inaccurate gps data, I have successfully modified my gps data in my bag file by assigning "-1" to status.status in sensor_msgs/NavSatStatus. And with the same bag only with configuration option use_nav_sat = false and use_nav_sat = true, I was able to produce the following maps.

Without gps, the map shows a failure in loop-closure (in red rectangle) that doesn't seem to be fixable by tuning the parameters:

While another run with gps (NavSatFix) shows that the area where loop-closure should happen seems to be fixable by tuning just a little bit more:

[lukehutch]

I am about to put together a Cartographer system (based on an Ouster OS-1^64 sensor), and have some related questions:

1. I plan to use a Here+ v2 GNSS sensor, which is based on the widely used u-blox M8N GNSS chip. Is this chip supported by Cartographer, or will I need to write some sort of driver for it?
2. The Here+ v2 also incorporates a 3-axis gyro, 3-axis accelerometer, and 3-axis magnetometer, based on the MPU9250. I don't know if this delivers raw signals or just the final post-Kalman-filtered results. Does anyone have experience with using this IMU with Cartographer?
3. The OS-1 LiDAR sensor also contains the same IMU chip, MPU9250. Is there any advantage to feeding redundant data from two IMU sensors into Cartographer? Is this even supported?
4. The Here+ v2 also contains an MS5611 barometer, which is important for the application I need cartographer for, since altitude measurements over long LiDAR traces need to be as accurate as possible. Does Cartographer support barometer inputs currently?
5. In the plots in the previous comment, @ludennis showed how loop closure did not seem to work within his original tolerance, even with GPS augmentation. @cschuet said "Right, GPS is only used in the global SLAM part, i.e. it enters the pose graph as constraints." Does that mean that GPS is only useful for actual loop closure, but doesn't help warp the relative positioning of SLAM to the absolute GNSS coordinate system if there are no large-scale loops?
6. If I also attach cameras to my scanning rig (either front and rear facing, or a 360 degree camera), and I want to colorize the point cloud, can Cartographer do that, or do I need to use some 3rd party software for this? If Cartographer can do it, how do I describe the geometry of the camera lens/FOV to Cartographer?
7. Can Cartographer produce a triangle mesh from the point cloud?

[gaschler]

@lukehutch

I am about to put together a Cartographer system (based on an Ouster OS-1^64 sensor), and have some related questions:

1. I plan to use a Here+ v2 GNSS sensor, which is based on the widely used u-blox M8N GNSS chip. Is this chip supported by Cartographer, or will I need to write some sort of driver for it?

Most users use cartographer_ros, for that you'd need a driver that publishes ROS IMU messages.

2. The Here+ v2 also incorporates a 3-axis gyro, 3-axis accelerometer, and 3-axis magnetometer, based on the MPU9250. I don't know if this delivers raw signals or just the final post-Kalman-filtered results. Does anyone have experience with using this IMU with Cartographer?

Only a partial answer.
Cartographer does its own special IMU integration that ignores motion in gravity direction,
which does well with raw data.
It is easy to experiment here and it should be worth it.
Timestamps must be in sync with the lidar.

3. The OS-1 LiDAR sensor also contains the same IMU chip, MPU9250. Is there any advantage to feeding redundant data from two IMU sensors into Cartographer? Is this even supported?

You can only input one IMU.

4. The Here+ v2 also contains an MS5611 barometer, which is important for the application I need cartographer for, since altitude measurements over long LiDAR traces need to be as accurate as possible. Does Cartographer support barometer inputs currently?

No, barometer data is not supported.

5. In the plots in the previous comment, @ludennis showed how loop closure did not seem to work within his original tolerance, even with GPS augmentation. @cschuet said "Right, GPS is only used in the global SLAM part, i.e. it enters the pose graph as constraints." Does that mean that GPS is only useful for actual loop closure, but doesn't help warp the relative positioning of SLAM to the absolute GNSS coordinate system if there are no large-scale loops?

Global SLAM is always done, even if there are no loops in the trajectory. GPS helps on the pose graph level but not inside a submap.

6. If I also attach cameras to my scanning rig (either front and rear facing, or a 360 degree camera), and I want to colorize the point cloud, can Cartographer do that, or do I need to use some 3rd party software for this? If Cartographer can do it, how do I describe the geometry of the camera lens/FOV to Cartographer?

That you would need to integrate yourself. There are no camera-related features.

7. Can Cartographer produce a triangle mesh from the point cloud?

No.

Hope this helps

BTW, you could have opened a new issue because you have questions about different hardware.

[lukehutch]

@gaschler Thank you for your answers.

BTW, you could have opened a new issue because you have questions about different hardware.

OK, I will open a separate issue for a followup question on the barometer specifically, but I'll ask a couple of quick followup questions here to keep the discussion together.

Global SLAM is always done, even if there are no loops in the trajectory. GPS helps on the pose graph level but not inside a submap.

It sounds like you are saying that the IMU helps during the SLAM stage, but GPS helps at the pose graph stage? Or are both IMU and GPS used at the pose graph stage? I am wondering if it is possible to bypass the IMU integration part of Cartographer entirely, and deliver already-integrated global position and pose estimates to the pose graph part of the pipeline, or if IMU and GPS signals should be split and provided separately to different stages. (They have very different sample rates anyway, so I'm guessing they are split.)

There are no camera-related features.

1. Can Cartographer produce a triangle mesh from the point cloud?

No.

(I'm a Xoogler) -- I believe Google has a couple of different internal libraries for meshing point clouds, and for colorizing point clouds from camera data -- is there a chance any of this could be open sourced and added to Cartographer? It would be a valuable addition.

[Fanny-one]

Hi @ludennis,
I am trying to use GPS to assist SLAM for 3D mapping of outdoor large scale environments and you have
tried it.
NOW I have employ nmea_navsat_driver get my gps topic, then what else should I do except assigning use_nav_sat = 1?
Besides how to assign "-1" to status.status in sensor_msgs/NavSatStatus?
Thank you very much!

[Fanny-one]

Hi @saimanoj18 @lukehutch ,
I'm planning to employ GPS/INS integrated navigation for FixedFramePoseData. From it I can get longtitude, latitude, attitude, roll, pitch and yaw so that FixedFramePoseData optimize translation and rotation.(#1154 (comment))
But how do I that?
Thank you very much!

[lukehutch]

@Fanny-one I don't know how to do this yet, but I will be attempting it over the next month with an Ouster LiDAR. Ouster directed me to this blog post written by one of their engineers. This post and others on the same blog talk about GPS integration, but I haven't looked closely yet:

https://www.wilselby.com/2019/06/ouster-os-1-lidar-and-google-cartographer-integration/

I have the same questions you do though. I hope it's relatively straightforward.

[Fanny-one]

@Fanny-one I don't know how to do this yet, but I will be attempting it over the next month with an Ouster LiDAR. Ouster directed me to this blog post written by one of their engineers. This post and others on the same blog talk about GPS integration, but I haven't looked closely yet:

Thank you for your answers.
I have read the blog you provide but they don't use GPS. thanks all the same.

[lukehutch]

@Fanny-one I don't know how to do this yet, but I will be attempting it over the next month with an Ouster LiDAR. Ouster directed me to this blog post written by one of their engineers. This post and others on the same blog talk about GPS integration, but I haven't looked closely yet:

Thank you for your answers.
I have read the blog you provide but they don't use GPS. thanks all the same.

You have to read the other posts in the same blog. There are at least one or two on using GPS. But I have yet to look closely at it.