No localization with Livox Lidar #5

WeilongXia · 2023-06-17T06:45:03Z

Thanks a lot for opening source such a great work, but when I run the code, I found that the odometry output pose has always been the origin and has not changed. I have checked the data topics of lidar and imu, and there is nothing wrong. The frequency of lidar is 10Hz, and the frequency of imu is 200Hz. What are the possible reasons for the above case? Looking forward to your reply~

kennyjchen · 2023-06-19T17:27:13Z

Yeah I haven't tested Livox support out since I don't have any data. Think you could upload a bag for me to play around with? Thanks!

WeilongXia · 2023-06-20T03:43:36Z

This is the dataset of Livox-mid360: https://1drv.ms/u/s!AjkIEg1kWUNeiAIZHIDBTzXmtHqf?e=cWek2u
And the extrinsic between the Lidar frame and IMU frame is

Joosoo1 · 2023-06-20T03:58:42Z

Yes, it seems that we are experiencing the same problem, may I ask if you are using it as a navigation for robots? Drones or unmanned vehicles?Maybe we can communicate that i have recently encountered some difficulties.

WeilongXia · 2023-06-20T09:04:04Z

Yeah I 'm going to use it as a localization module for drones, and maybe we can use the g-mail to communicate: xiawl0524@gmail.com

Yes, it seems that we are experiencing the same problem, may I ask if you are using it as a navigation for robots? Drones or unmanned vehicles?Maybe we can communicate that i have recently encountered some difficulties.↳

Joosoo1 · 2023-06-21T08:51:53Z

ok, after my test, livox-mid360 can run DLIO successfully, just change the point cloud publishing form "xfer_format" to "2" in livox_ros_driver2, then it can run DLIO successfully. DLIO can be run successfully, as follows:

Joosoo1 · 2023-06-21T08:57:01Z

But the results do not seem to be very good, odom data z-axis jitter will be relatively large, may be due to the reason of not tuning parameters, but also may be due to the built-in imu noise is too large, the follow-up we can exchange about how to make it better and more stable, in addition, I am also intended to apply it to the top of the drone, used as navigation and positioning.

WeilongXia · 2023-06-21T09:17:44Z

Thanks to your reply, the dimension of the acceleration data from the build-in imu in Livox MID-360 is g(gravity) instead of m/s^2, maybe you should multiply the data with a constant value like 9.8?

Joosoo1 · 2023-06-21T09:51:30Z

Thanks to your reply, the dimension of the acceleration data from the build-in imu in Livox MID-360 is g(gravity) instead of m/s^2, maybe you should multiply the data with a constant value like 9.8?

ok,i will try!If you have better results, please let me know too!

WeilongXia · 2023-06-21T12:14:30Z

I have tried to set xfer_format t o 2, and the result seems great!

Joosoo1 · 2023-06-21T14:02:21Z

wow! Your results look great, the position estimation is accurate and the built point cloud map has no drift, but my results are as bad as ever, still with a large offset on the z-axis, can you share your configuration parameters file? Also, about the z-axis, my livox has a z-axis accel bias of -5, while yours is 0. I think there is something wrong here.

kennyjchen · 2023-06-21T17:02:40Z

Hi @WeilongXia and @Joosoo1 -- thanks for the great discussion you two have started here. Looks like the Livox branch is working! It just needs to be published in the standard format.

Yeah something is definitely wrong if the IMU biases are that large. Have you checked your extrinsics? That's typically an issue, especially since the convention for IMU and LiDAR are different.

Edit: it's working, but deskewing is not (note the deskewed points: 0 in the terminal screenshots above). Looks like a per-point timestamp issue with Livox sensors.

kennyjchen · 2023-06-21T21:32:56Z

@Joosoo1 Yes, change the units of the accelerometer to m/s^2 as per the advice above.

Could someone record me a dataset in the new xfer_format? Thanks.

Joosoo1 · 2023-06-23T05:40:45Z

Well, it's true that because of the acceleration units, it has to be m/s^2 for it to work well.
I provide the dataset：https://1drv.ms/u/s!AiyqJY-cvK5varx4IwdfshuTfeE?e=TtbAdI
pointcloud_topic:/livox/lidar
imu_topic:/livox/imu (the acceleration unit is g, so you need to modify the code in DLIO)
The parameters are:
! livox-mid360 (built-in imu) extranet

dxdydz3dt · 2023-06-23T14:43:24Z

Can you translate in English Please!. I am trying to run Livox but its not working

Joosoo1 · 2023-06-23T15:11:23Z

Can you translate in English Please!. I am trying to run Livox but its not working
You can take a look at this: #5 (comment)

It is necessary to additionally convert the acceleration units of the /livox/imu topic output to m/s^2,then the DLIO will work well, of course all this needs to be under the source code of the livox branch.

Joosoo1 · 2023-06-23T15:16:19Z

@kennyjchen I found that when livox outputs the standard point cloud, its feild.name is not "offset_time", so according to the code "deskew_= false", deskewing is not wrok!

kennyjchen · 2023-06-23T18:47:04Z

Hmm, yeah unfortunately it looks like in the livox_ros_driver2 package, only x y z and intensity fields are copied over when publishing using the xfer_format: 2 value.

https://github.com/Livox-SDK/livox_ros_driver2/blob/master/src/lddc.cpp#L437

It looks like with their custom message, they have per-point timestamps . However, it looks like both timestamp and offset_time are used, which is a bit confusing since their CustomPoint.msg contains offset_time, but the data above (the first link) has a timestamp field.

If only timestamps are available using their custom point format, we'll need to build support for that (unless they add timestamps to the PCL format). I'll keep digging.

kennyjchen · 2023-06-24T00:51:12Z

Ok -- I think I might have figured this out. For future reference, this is how different sensors timestamp their points:

Ouster
- field t of type std::uint32_t, which is the time relative to the beginning of the scan in nanoseconds
Velodyne
- field time of type float, which is the time relative to the beginning of the scan in seconds
Livox
- xfer_format: 0: field timestamp of type double, which is the absolute time of the point since epoch in nanoseconds
- xfer_format: 1: field offset_time of type std::uint32_t, which is the time relative to the beginning of the scan in nanoseconds
Hesai
- field timestamp of type double, which is the absolute time of the point since epoch in seconds

I pushed a few changes to the feature/livox-support branch. Please test this out using the original xfer_format value so that there are timestamp fields.

@WeilongXia -- I played around with your data, and it looks like the LiDAR and IMU measurements are not time synchronized. Is there an option in the Livox driver to synchronize them somehow?

EDIT 6/28: updated the list above to be more accurate and include both xfer_format: 0 and xfer_format: 1. As of the time of this writing, seems like there is a bug in timestamping with xfer_format: 0.

WeilongXia · 2023-06-24T03:30:46Z

Ok, I found a timesync choice in the config file like this

and I'll try to record a new rosbag.

juliangaal · 2023-06-24T10:07:14Z

@WeilongXia could you rename the issue to something more meaningful, if you don't mind, for future reference? Something like "No localization with Livox Lidar"

WeilongXia · 2023-06-25T02:48:06Z

I tested the feature/livox-support branch with the original xfer_format(value 1) , but there is no localization output. @Joosoo1 Have you tested the new branch with xfer_format value is 1?
In livox_ros_driver2, when I run the msg_MID360.launch file, the lidar topic type is livox_ros_driver2/CustomMsg rather than sensor_msgs::PointCloud2, so I modify some code in livox_ros_driver2 like this:

else if (kLivoxCustomMsg == transfer_format_)
        {
            // PublishCustomPointcloud(p_queue, index);
            PublishPointcloud2(p_queue, index);
        }

else if (kLivoxCustomMsg == transfer_format_)
        {
            // **pub = cur_node_->GetNode().advertise<livox_ros_driver2::CustomMsg>(name_str, queue_size);
            // DRIVER_INFO(*cur_node_, "%s publish use livox custom format, set ROS publisher queue size %d", name_str,
            //             queue_size);
            **pub = cur_node_->GetNode().advertise<sensor_msgs::PointCloud2>(name_str, queue_size);
            DRIVER_INFO(*cur_node_, "%s publish use PointCloud2 format, set ROS publisher queue size %d", name_str,
                        queue_size);
        }

else if (kLivoxCustomMsg == msg_type)
    {
        // DRIVER_INFO(*cur_node_, "%s publish use livox custom format", topic_name.c_str());
        // return cur_node_->create_publisher<CustomMsg>(topic_name, queue_size);
        DRIVER_INFO(*cur_node_, "%s publish use PointCloud2 format", topic_name.c_str());
        return cur_node_->create_publisher<PointCloud2>(topic_name, queue_size);
    }

yijing2050 · 2023-06-26T02:45:36Z

ld multiply the data with a constant value like 9.8?

@Joosoo1 I used xfer_format = 0,and there was no point cloud. My wechat:yijingxs, you can add me and it is convenient to communicate with each other

Joosoo1 · 2023-06-26T03:27:04Z

Yes, I tested the new branch and the DLIO only has odom output when xfer_format=2, but the result is completely wrong, so I have been using the original DLIO now, the original DLIO works well, I tested it on the drone and the positioning is very good. But deskwing is not working and will have some effect on DLIO，So I look forward to the author's help in solving this problem.

Joosoo1 · 2023-06-26T03:28:50Z

ld multiply the data with a constant value like 9.8?

@Joosoo1 I used xfer_format = 0,and there was no point cloud. My wechat:yijingxs, you can add me and it is convenient to communicate with each other
xfer_format = 2, old DLIO could working!

Joosoo1 · 2023-06-26T03:54:34Z

ld multiply the data with a constant value like 9.8?

@Joosoo1 I used xfer_format = 0,and there was no point cloud. My wechat:yijingxs, you can add me and it is convenient to communicate with each other
xfer_format = 2, old DLIO could working!

Also, when you use imu with acceleration units of m/s^2, you should change the https://github.com/vectr-ucla/direct_lidar_inertial_odometry/tree/feature/livox-support/src/ under the livox branch Lines 1373 to 1375 of the code in dlio/odom.cc should be changed to Eigen::Vector3f lin_accel(imu_raw->linear_acceleration.x * gravity_, imu_raw->linear_acceleration.y * gravity_, imu_raw->linear_acceleration.z * gravity_); recompile！

yijing2050 · 2023-06-26T04:25:50Z

Thanks for your reply and kind advice. I have another problem when compiling, the first is the version of cmake. As my ros version is melodic,and the cmake version is 3.10.2, so I modify #VERSION 3.12.4 cmake_minimum_required(VERSION 3.10.2) project(direct_lidar_inertial_odometry) Could this modification do influence the output of the code?  it have no add_compile_definitions.Then I comment out the section:# Not all machines have <cpuid.h> available #set(CMAKE_TRY_COMPILE_TARGET_TYPE STATIC_LIBRARY) #file(WRITE ${CMAKE_BINARY_DIR}/test_cpuid.cpp "#include <cpuid.h>") #try_compile(HAS_CPUID ${CMAKE_BINARY_DIR} ${CMAKE_BINARY_DIR}/test_cpuid.cpp) #file(REMOVE ${CMAKE_BINARY_DIR}/test_cpuid.cpp) #if(HAS_CPUID) #  add_compile_definitions(HAS_CPUID) #endif() after done above, I compiled successfully.

…

------------------ 原始邮件 ------------------ 发件人: ***@***.***>; 发送时间: 2023年6月26日(星期一) 中午11:54 收件人: ***@***.***>; 抄送: ***@***.***>; ***@***.***>; 主题: Re: [vectr-ucla/direct_lidar_inertial_odometry] No localization with Livox Lidar (Issue #5) ld multiply the data with a constant value like 9.8? @Joosoo1 I used xfer_format = 0,and there was no point cloud. My wechat:yijingxs, you can add me and it is convenient to communicate with each other xfer_format = 2, old DLIO could working! Also, when you use imu with acceleration units of m/s^2, you should change the https://github.com/vectr-ucla/direct_lidar_inertial_odometry/tree/feature/livox-support/src/ under the livox branch Lines 1373 to 1375 of the code in dlio/odom.cc should be changed to Eigen::Vector3f lin_accel(imu_raw->linear_acceleration.x * gravity_, imu_raw->linear_acceleration.y * gravity_, imu_raw->linear_acceleration.z * gravity_); recompile！ — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: ***@***.***>

Joosoo1 · 2023-06-26T07:05:51Z

Thanks for your reply and kind advice. I have another problem when compiling, the first is the version of cmake. As my ros version is melodic,and the cmake version is 3.10.2, so I modify #VERSION 3.12.4 cmake_minimum_required(VERSION 3.10.2) project(direct_lidar_inertial_odometry) Could this modification do influence the output of the code? it have no add_compile_definitions.Then I comment out the section:# Not all machines have <cpuid.h> available #set(CMAKE_TRY_COMPILE_TARGET_TYPE STATIC_LIBRARY) #file(WRITE ${CMAKE_BINARY_DIR}/test_cpuid.cpp "#include <cpuid.h>") #try_compile(HAS_CPUID ${CMAKE_BINARY_DIR} ${CMAKE_BINARY_DIR}/test_cpuid.cpp) #file(REMOVE ${CMAKE_BINARY_DIR}/test_cpuid.cpp) #if(HAS_CPUID) # add_compile_definitions(HAS_CPUID) #endif() after done above, I compiled successfully.
…
------------------ 原始邮件 ------------------ 发件人: @.>; 发送时间: 2023年6月26日(星期一) 中午11:54 收件人: @.>; 抄送: @.>; @.>; 主题: Re: [vectr-ucla/direct_lidar_inertial_odometry] No localization with Livox Lidar (Issue #5) ld multiply the data with a constant value like 9.8? @Joosoo1 I used xfer_format = 0,and there was no point cloud. My wechat:yijingxs, you can add me and it is convenient to communicate with each other xfer_format = 2, old DLIO could working! Also, when you use imu with acceleration units of m/s^2, you should change the https://github.com/vectr-ucla/direct_lidar_inertial_odometry/tree/feature/livox-support/src/ under the livox branch Lines 1373 to 1375 of the code in dlio/odom.cc should be changed to Eigen::Vector3f lin_accel(imu_raw->linear_acceleration.x * gravity_, imu_raw->linear_acceleration.y * gravity_, imu_raw->linear_acceleration.z * gravity_); recompile！ — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.***>

I don't know much about this, but if it compiles successfully, you should be able to run the program successfully, so you can test it as soon as possible!

kennyjchen · 2023-06-26T17:47:23Z

@yijing2050 Unfortunately DLIO only supports ROS Noetic, but it sounds like you got it working. Removing those lines won't have any affect on the output; it's just to build support for ARM architectures.

@WeilongXia @Joosoo1 The xfer_format should be 0 so that it's a message of type PointCloud2 but contains the timestamps. I will look into this again later this week. Sorry to ask again -- could someone record me a new bag that ideally time synchronizes the Mid-360's LiDAR and IMU (if not already), using xfer_format: 0? Thank you. In the meanwhile, you can turn deskewing off, but note that (especially at 10Hz) the point clouds will be quite distorted.

yijing2050 · 2023-06-27T02:00:26Z

@kennyjchen
livox_repub.zip
Uploading livox_repub.zip…

yijing2050 · 2023-06-27T02:22:30Z

Actually, I have compiled successfully although, but i haven't got the odom.. The configuration is as the below.

kennyjchen · 2023-06-27T22:47:13Z

Ok -- after digging through the Livox driver and also looking at this dataset, it seems like there is a bug for per-point timestamps when using xfer_format: 0(i.e., Livox pointcloud2(PointXYZRTLT) pointcloud format). The point cloud and IMU timestamps look synchronized, but the point timestamps are off and do not fall within the sweep window for some reason. This might be just the provided dataset, but either way they don't look lined-up.

I looked at other Mid-360 packages (e.g., this), and it seems like most people use xfer_format: 1 (Livox's custom pointcloud format) so that the offset_time field is available and contains the time since the beginning of the sweep. This requires a subscription to Livox's custom pointcloud format though.

So new plan: please try the updated branch but with xfer_format: 1. Your topic of type livox_ros_driver::CustomMsg should be called /robot/livox, which will call this new callback. This is all assuming that offset_time is in nanoseconds relative to the beginning of the sweep (which I think is the case).

(Please be careful with the units of gravity. I removed the previous multiplication with gravity.)

Note that I have not tested this at all, because I do not have a Livox sensor or any bags with xfer_format: 1. If someone could record me one that would be great so I can test it on my side. Otherwise, let me know if it works or not!

Joosoo1 · 2023-06-28T15:44:10Z

Sorry for the late reply, and thank you @kennyjchen author for your efforts to solve this problem. Here is the dataset I recorded: accel in units is g；and xfer_format: 1
https://onedrive.live.com/?cid=6FAEBC9C8F25AA2C&id=6FAEBC9C8F25AA2C%21120&parId=root&o=OneUp

kennyjchen · 2023-06-28T18:26:29Z

Ok, I think I got it working. Credit to this Tweet for supplying the data I played around with.

Please use the newer livox_ros_driver2 package with xfer_format: 1. The latest feature/livox-support branch seems to be working with deskewing. Let me know.

Joosoo1 · 2023-07-02T15:08:56Z

Ok, I think I got it working. Credit to this Tweet for supplying the data I played around with.

Please use the newer livox_ros_driver2 package with xfer_format: 1. The latest feature/livox-support branch seems to be working with deskewing. Let me know.

Doesn't seem to work：

WeilongXia · 2023-07-03T02:34:08Z

Ok, I think I got it working. Credit to this Tweet for supplying the data I played around with.↳

Please use the newer livox_ros_driver2 package with xfer_format: 1. The latest feature/livox-support branch seems to be working with deskewing. Let me know.↳

Doesn't seem to work： ↳

You should modify livox_topic to "/livox/lidar", and pointcloud_topic to another one.
Hi author @kennyjchen , with deskew set to "false", it can work with Livox-MID360 now as below!

And the offset_time units in livox_ros_driver2 is microseconds.

WeilongXia · 2023-07-03T08:38:34Z

Hi author, when i set deskew to true, the program is not working properly, this is the dataset i used: https://1drv.ms/u/s!AjkIEg1kWUNeiArwedvaRRYd5hWB?e=BaII3v
Could you please help to figure this out?

kennyjchen · 2023-07-03T17:00:07Z

@WeilongXia -- Your dataset works for me with deskewing.

I'm using the latest feature/livox-support branch without any modifications and the following command:

rosbag play 2023-06-28-10-00-30.bag /livox/imu:=/robot/imu /livox/lidar:=/robot/livox

What are the errors that you're seeing?

Joosoo1 · 2023-07-04T02:49:35Z

@@yES! DLIO is now working with deskewing！just modify livox_topic to "/livox/lidar"

kennyjchen · 2023-07-05T17:49:15Z

Good to hear!

yijing2050 · 2023-07-06T14:34:00Z

DLIO is now working with deskewing in Ubuntu 18.04, but it seems not stable and the path is not smooth as fast_lio2. And the map will fly in some time later.

kennyjchen · 2023-07-11T17:41:02Z

That max comp time looks pretty awful, which can result in these sorts of behaviors if too many frames are skipped. Are extrinsics correct?

juliangaal · 2023-07-11T19:43:15Z

@Joosoo1 what are the extrinsics of the dataset you provided?

Joosoo1 · 2023-07-12T03:01:49Z

@Joosoo1 what are the extrinsics of the dataset you provided?
Please use this dataset, I tested it and it works great!：https://1drv.ms/u/s!AjkIEg1kWUNeiArwedvaRRYd5hWB?e=BaII3v
The extrinsics are as follows：

CharlieV5 · 2023-07-17T16:05:36Z

May I ask you guys what is the meaning of dlio/pointcloud/dekew in dlio.yaml? Does it mean that this program will do the deskewing for pointcloud, or this program will consider the input points has been deskewed so it will no do the deskewing? What will happen when dlio/pointcloud/dekew is set as false? Thank you!

Ok, I think I got it working. Credit to this Tweet for supplying the data I played around with.↳

Please use the newer livox_ros_driver2 package with xfer_format: 1. The latest feature/livox-support branch seems to be working with deskewing. Let me know.↳

Doesn't seem to work： ↳

You should modify livox_topic to "/livox/lidar", and pointcloud_topic to another one. Hi author @kennyjchen , with deskew set to "false", it can work with Livox-MID360 now as below! And the offset_time units in livox_ros_driver2 is microseconds.

kennyjchen · 2023-07-17T21:03:33Z

It will undistort the point cloud when set to true.

CharlieV5 · 2023-07-18T02:51:43Z

It will undistort the point cloud when set to true.

Thanks

…-to-lidar updated interfaces

WeilongXia changed the title ~~the output has been unchanged~~ No localization with Livox Lidar Jun 24, 2023

kennyjchen closed this as completed Jul 18, 2023

schablonmassig mentioned this issue Mar 23, 2024

About livox lidar davidskdds/DMSA_LiDAR_SLAM#3

Closed

gandres42 pushed a commit to Cardinal-Space-Mining/direct_lidar_inertial_odometry that referenced this issue May 13, 2024

Merge pull request vectr-ucla#5 from Cardinal-Space-Mining/merge-main…

c8fd9e5

…-to-lidar updated interfaces

No localization with Livox Lidar #5

No localization with Livox Lidar #5

Comments

WeilongXia commented Jun 17, 2023

kennyjchen commented Jun 19, 2023

WeilongXia commented Jun 20, 2023

Joosoo1 commented Jun 20, 2023

WeilongXia commented Jun 20, 2023

Joosoo1 commented Jun 21, 2023

Joosoo1 commented Jun 21, 2023

WeilongXia commented Jun 21, 2023

Joosoo1 commented Jun 21, 2023

WeilongXia commented Jun 21, 2023

Joosoo1 commented Jun 21, 2023

kennyjchen commented Jun 21, 2023 • edited Loading

kennyjchen commented Jun 21, 2023

Joosoo1 commented Jun 23, 2023 • edited Loading

dxdydz3dt commented Jun 23, 2023

Joosoo1 commented Jun 23, 2023

Joosoo1 commented Jun 23, 2023 • edited Loading

kennyjchen commented Jun 23, 2023

kennyjchen commented Jun 24, 2023 • edited Loading

WeilongXia commented Jun 24, 2023

juliangaal commented Jun 24, 2023

WeilongXia commented Jun 25, 2023 • edited Loading

yijing2050 commented Jun 26, 2023

Joosoo1 commented Jun 26, 2023

Joosoo1 commented Jun 26, 2023

Joosoo1 commented Jun 26, 2023

yijing2050 commented Jun 26, 2023 via email

Joosoo1 commented Jun 26, 2023 • edited Loading

kennyjchen commented Jun 26, 2023

yijing2050 commented Jun 27, 2023 • edited Loading

yijing2050 commented Jun 27, 2023

kennyjchen commented Jun 27, 2023 • edited Loading

Joosoo1 commented Jun 28, 2023 • edited Loading

kennyjchen commented Jun 28, 2023 • edited Loading

Joosoo1 commented Jul 2, 2023

WeilongXia commented Jul 3, 2023

WeilongXia commented Jul 3, 2023

kennyjchen commented Jul 3, 2023

Joosoo1 commented Jul 4, 2023 • edited Loading

kennyjchen commented Jul 5, 2023

yijing2050 commented Jul 6, 2023

kennyjchen commented Jul 11, 2023

juliangaal commented Jul 11, 2023

Joosoo1 commented Jul 12, 2023

CharlieV5 commented Jul 17, 2023

kennyjchen commented Jul 17, 2023

CharlieV5 commented Jul 18, 2023

kennyjchen commented Jun 21, 2023 •

edited

Loading

Joosoo1 commented Jun 23, 2023 •

edited

Loading

Joosoo1 commented Jun 23, 2023 •

edited

Loading

kennyjchen commented Jun 24, 2023 •

edited

Loading

WeilongXia commented Jun 25, 2023 •

edited

Loading

Joosoo1 commented Jun 26, 2023 •

edited

Loading

yijing2050 commented Jun 27, 2023 •

edited

Loading

kennyjchen commented Jun 27, 2023 •

edited

Loading

Joosoo1 commented Jun 28, 2023 •

edited

Loading

kennyjchen commented Jun 28, 2023 •

edited

Loading

Joosoo1 commented Jul 4, 2023 •

edited

Loading