-
Notifications
You must be signed in to change notification settings - Fork 0
Benchmarks LIO
If You are looking for end-to-end mobile mapping framework You can start from this project. We provide a benchmark for other approaches: RKO-LIO (2026), R-VoxelMap (2026), BIEVR-LIO (2026), SE3-LIO (2026), EllipseLIO(2026), D-LIO (2026), FORM (2026), Super-LIO (2026), DALI_SLAM (2025), lidar_odometry_ros_wrapper (2025), SuperOdometry (2021), mola_lidar_odometry (2025), RESPLE (2025), GenZ-ICP (2025), MM-LINS (2025), Voxel-SLAM (2024), NV-LIOM (2024), c3p-voxelmap (2024), I2EKF-LO (2024), LIO-EKF (2024), iG-LIO (2024), GLIM (2024), Point-LIO (2024), LOG-LIO2 (2024), DLIO (2023), SLICT (2023), RA-L, KISS-ICP (2023), Faster-LIO (2022), VoxelMap (2022), DLO (2022), CT-ICP (2021), FAST-LIO (2020), LOAM-Livox (2019), LeGO-LOAM (2018). Our goal is to find/develop/discover best end-to-end mobile mapping framework. To cite benchmark suite please use as follows:
@article{BEDKOWSKI2026102822,
title = {MapsHD: A benchmark suite for LiDAR odometry frameworks},
journal = {SoftwareX},
volume = {35},
pages = {102822},
year = {2026},
issn = {2352-7110},
doi = {https://doi.org/10.1016/j.softx.2026.102822},
url = {https://www.sciencedirect.com/science/article/pii/S2352711026003146},
author = {Janusz Bȩdkowski and Michał Pełka and Karol Majek and Marcin Matecki and Adrian Radulescu and Charles Hamesse and Ethan Decleyn and Przemysław Lekston and Tomasz Owerko and Przemysław Kuras and Michał Ciszewski and Jakub Kolecki and Karolina Tomaszkiewicz and Łukasz Ambroziński and Joanna Koszyk and Bartosz Hyla and Karolina Pargieła and Anna Malczewska and Tomasz Lipecki and Artur Adamek and Bartosz Mitka and Klapa Przemysław and Pelagia Gawronek and Martin Mokros and Jozef Výboštok and Juliána Chudá and Michal Skladan and Carlos Cabo and Kim André Anstensen and Craciun Daniel-Marian and Antun Jakopec and Michal Wlasiuk and Kornel Mrozowski and Maksymilian Kulicki and Krzysztof Stereńczak and Oskar Bartosz and Jakub Markiewicz and Sławomir Łapiński and Adam Kostrzewa and Mariana Campos and Machi Zawidzki and Jacek Szklarski and Rami Faraj and Loris Redovniković and Jurica Jagetić and Samer Karam and Răzvan Dumbravă and Milosz Mielcarek and Grzegorz Krok and Michal Laszkowski and Jaroslaw Wajs and Jakub Chudziński},
keywords = {LiDAR odometry, LiDAR-inertial odometry, Benchmarking},
abstract = {This paper describes a software toolbox for LiDAR (Light Detection and Ranging) and LiDAR-Inertial Odometry qualitative and quantitative evaluation. We provide software as https://github.com/MapsHD organization with all necessary information at https://github.com/MapsHD/HDMapping. Our software contributions are a) ground truth data processing tool, b) dockerized state-of-the-art LO and LIO algorithms, c) multi-session data registration to common coordinate system, d) Absolute Pose Error (APE) and Relative Pose Error (RPE) metrics, e) import/export tools for easier 3D data handling and visualizing, e.g., in Cloud Compare software. This software is compatible with ROS1 (Robot Operating System) and ROS2 data formats. We show an example benchmark of LeGO-LOAM, LIO-SAM, FAST-LIO, DLO, VoxelMap, Faster-LIO, KISS-ICP, CT-ICP, SLICT, DLIO, GLIM, iG-LIO, LIO-EKF, I2EKF-LO, GenZ-ICP, RESPLE, odometry_ros_wrapper, Point-LIO, and LOAM-Livox algorithms. For all experiments we provide movies. The contribution of the paper is software-oriented LO/LIO algorithm benchmark suite. The novelty lies in the integration of multiple benchmarking steps into a unified framework, thus overall effort needed for qualitative and quantitative evaluation is reduced.}
}
Download the dataset from Bunker DVI Dataset

We provide HDMapping-LIO: easy to run, easy to test LiDAR Inertial Odometry that is as accurate as FAST-LIO, FASTER-LIO and much more precise movie.
All following algorithms are generating session compatible with 'multi_view_tls_registration_step_2'. It is designed for a benchmark.
- https://github.com/MapsHD/benchmark-HDMapping-ground-truth (2026, 'under review', movie)
- [1] https://github.com/MapsHD/benchmark-HDMapping_LIO-to-HDMapping (2026, 'under review', movie)
- [2] https://github.com/MapsHD/benchmark-rko_lio-to-HDMapping (2026, RA-L, BIB, movie ToDo)
- [3] https://github.com/MapsHD/benchmark-R-VoxelMap-to-HDMapping (2026, RA-L, BIB, movie ToDo)
- [4] https://github.com/MapsHD/benchmark-BIEVR-LIO-to-HDMapping (2026, arXiv, BIB, movie ToDo)
- [5] https://github.com/MapsHD/benchmark-SE3-LIO-to-HDMapping (2026, ICRA, BIB, movie ToDo)
- [6] https://github.com/MapsHD/benchmark-EllipseLIO-to-HDMapping (2026, arXiv, BIB, movie ToDo)
- [7] https://github.com/MapsHD/benchmark-D-LIO-to-HDMapping (2026, RA-L, BIB, movie ToDo)
- [8] https://github.com/MapsHD/benchmark-FORM-to-HDMapping (2026, ICRA, BIB, movie)
- [9] https://github.com/MapsHD/benchmark-Super-LIO-to-HDMapping (2026, RA-L, BIB, movie)
- [10] https://github.com/MapsHD/benchmark-DALI_SLAM-to-HDMapping (2025, Elsevier ISPRS, BIB)
- [11] https://github.com/MapsHD/benchmark-lidar_odometry_ros_wrapper-to-HDMapping (2025, RA-L, BIB, movie)
- [12] https://github.com/MapsHD/benchmark-SuperOdometry-to-HDMapping (2021, IROS, 2025, ICRA, BIB, movie)
- [13] https://github.com/MapsHD/benchmark-mola_lidar_odometry-to-HDMapping (2025, IJRR, BIB, movie)
- [14] https://github.com/MapsHD/benchmark-RESPLE-to-HDMapping (2025, RA-L, BIB, movie) (ToDo --> add to automated benchmark)
- [15] https://github.com/MapsHD/benchmark-GenZ-ICP-to-HDMapping (2025, RA-L, BIB, movie)
- [16] https://github.com/MapsHD/benchmark-MM-LINS-to-HDMapping (2025, TIV, BIB, movie ToDo)
- [17] https://github.com/MapsHD/benchmark-PIN-SLAM-to-HDMapping (2024, TRO, BIB, movie ToDo)
- [18] https://github.com/MapsHD/benchmark-SR-LIO-to-HDMapping (2024, IROS, BIB, movie ToDo)
- [19] https://github.com/MapsHD/benchmark-Voxel-SLAM-to-HDMapping (2024, arXiv, BIB, 2nd place ICRA HILTI 2023 SLAM Challenge, 1st place ICCV 2023 SLAM Challenge, movie ToDo)
- [20] https://github.com/MapsHD/benchmark-NV-LIOM-to-HDMapping (2024, RA-L, BIB, movie ToDo)
- [21] https://github.com/MapsHD/benchmark-c3p-voxelmap-to-HDMapping (2024, arXiv, BIB, movie ToDo)
- [22] https://github.com/MapsHD/benchmark-I2EKF-LO-to-HDMapping (2024, arXiv, BIB, movie)
- [23] https://github.com/MapsHD/benchmark-LIO-EKF-to-HDMapping (2024, ICRA, BIB, movie)
- [24] https://github.com/MapsHD/benchmark-iG-LIO-to-HDMapping (2024, RA-L, BIB, movie)
- [25] https://github.com/MapsHD/benchmark-GLIM-to-HDMapping (2024, arXiv, BIB, movie)
- [26] https://github.com/MapsHD/benchmark-Point-LIO-to-HDMapping (2024, JAIS, BIB, movie)
- [27] https://github.com/MapsHD/benchmark-LOG-LIO2-to-HDMapping (2024, arXiv, BIB, movie ToDo)
- [28] https://github.com/MapsHD/benchmark-PV-LIO-to-HDMapping (2023, no publication, movie ToDo)
- [29] https://github.com/MapsHD/benchmark-DLIO-to-HDMapping (2023, ICRA, BIB, movie)
- [30] https://github.com/MapsHD/benchmark-SLICT-to-HDMapping (2023, RA-L, BIB, movie)
- [31] https://github.com/MapsHD/benchmark-KISS-ICP-to-HDMapping (2023, RA-L, BIB, movie)
- [32] https://github.com/MapsHD/benchmark-Faster-LIO-to-HDMapping (2022, RA-L, BIB, movie)
- [33] https://github.com/MapsHD/benchmark-VoxelMap-to-HDMapping (2022, arXiv, BIB, movie)
- [34] https://github.com/MapsHD/benchmark-DLO-to-HDMapping (2022, RA-L, BIB, movie)
- [35] https://github.com/MapsHD/benchmark-CT-ICP-to-HDMapping (2021, arXiv, BIB, movie)
- [36] https://github.com/MapsHD/benchmark-FAST-LIO-to-HDMapping (2020, arXiv, BIB, movie)
- [37] https://github.com/MapsHD/benchmark-LOAM-Livox-to-HDMapping (2019, arXiv, BIB, movie)
- [38] https://github.com/MapsHD/benchmark-LeGO-LOAM-to-HDMapping (2018, IROS, BIB, movie)
qualitative evaluation benchmark movie quantitative evaluation benchmark movie quantitative benchmark
Benchmark Orchestration
Datasets
- https://charleshamesse.github.io/bunker-dvi-dataset/
- https://github.com/Jakubach/kitti_to_ros (Michal W)
- https://github.com/Jakubach/kitti_to_hdmapping (Michal W)
- M2DGR https://github.com/SJTU-ViSYS/M2DGR
- M3DSS https://neufs-ma.github.io/M3DSS/index.html (benchmark, dataset)
- NCLT https://robots.engin.umich.edu/nclt/
- Hilti SLAM Challenge https://hilti-challenge.com/dataset-2023 (Michal P)
- Complex Urban https://sites.google.com/view/complex-urban-dataset
- NTU VIRAL https://ntu-aris.github.io/ntu_viral_dataset/
- HeRCULES
- He-LiPR
- AevaScene
- https://thisparticle.github.io/geode/
Algorithms without code
- https://kafeiyin00.github.io/AEOS/ (2026, ISPRS-Elsevier, BIB, no source code)
- VOX-LIO https://www.mdpi.com/2072-4292/17/13/2214 (2025, MDPI Remote Sensing, BIB, no source code)
- SS-LIO: Robust Tightly Coupled Solid-State LiDAR–Inertial Odometry for Indoor Degraded Environments (2025, MDPI electronics, BIB, no source code)
- https://www.sciencedirect.com/science/article/abs/pii/S1566253525002052 (2025, Elsevier Information Fusion, BIB, no source code)
- https://ieeexplore.ieee.org/abstract/document/11206445 (2025, RA-L, BIB, no source code)
- https://ieeexplore.ieee.org/abstract/document/11045969 (2025, RA-L, BIB, no source code)
- https://ieeexplore.ieee.org/document/10954274 (2025, IEEE TITS, BIB, no source code)
- https://github.com/kafeiyin00/HCTO (2024, ISPRS-Elsevier, BIB, no source code)
- Fast and Robust LiDAR-Inertial Odometry by Tightly-Coupled Iterated Kalman Smoother and Robocentric Voxels, RC-Vox (2024, IEEE TITS, BIB, no source code)
- https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=11419773 (2026, RA-L, BIB), https://github.com/JMUWRobotics/Dynamic-ICP
Algorithms with unsolved issues
- https://github.com/MapsHD/benchmark-KISS-SLAM-to-HDMapping (2025, IROS, BIB) (not solved issue https://github.com/kpmrozowski/KISS-SLAM-to-HDMapping/issues/1)
- https://github.com/MapsHD/rko_lio (2025, arXiv, BIB) (not solved issue: https://github.com/marcinmatecki/rko-lio-to-HDMapping/issues/1)
- https://github.com/MapsHD/benchmark-MAD-ICP-to-HDMapping (2024, RA-L, BIB, compatibilty issue - only trajectory provided, repository inactive)
- https://github.com/Ji1Xingyu/lio_gvm (2024, RA-L, BIB) (not solved issues: https://github.com/Ji1Xingyu/lio_gvm/issues/13, https://github.com/Ji1Xingyu/lio_gvm/issues/12, https://github.com/Ji1Xingyu/lio_gvm/issues/7)
- https://github.com/thisparticle/btsa (2025, RA-L, BIB, dynamic scenes) unsolved issue https://github.com/thisparticle/btsa/issues/4
- https://github.com/BrenYi/Light-LOAM (2024, RA-L, BIB) (issue) https://github.com/BrenYi/Light-LOAM/issues/6
- https://github.com/clegenti/2fast2lamaa (2025, arXiv, BIB) (issue) https://github.com/clegenti/2fast2lamaa/issues/4
- https://github.com/Livox-SDK/LIO-Livox (2026, Livox proprietary, BIB) (doesn't support non-repetitive lidar https://github.com/Livox-SDK/LIO-Livox/issues/81)
Not supported algorithm
- https://ieeexplore.ieee.org/document/10900461, https://github.com/kafeiyin00/UA-MPC (2025, RA-L, BIB, we do not support rotary LiDAR)
- https://github.com/YangSiri/OR-LIM (2022, we do not support rotary LiDAR)
- https://github.com/MapsHD/benchmark-LiDAR-IMU-Init-to-HDMapping (2022) (extrinsic calibration)
Required PCD format:
- https://github.com/KTH-RPL/dufomap (2024, RA-L, BIB, dynamic scenes)
- https://github.com/KTH-RPL/DynamicMap_Benchmark (2023, ITSC, BIB, dynamic scenes)
ToDo Build Success:
- https://github.com/SlamCabbage/Optimized-SC-F-LOAM (2022, CVCI, BIB) - Marcin (Scan Context-based loop closure)
- https://github.com/wh200720041/floam (2021, IROS, BIB) - Piotr 1
- https://github.com/MapsHD/benchmark-LIO-SAM-to-HDMapping (2020, IROS, BIB, with loop closures) - Piotr 2
- https://github.com/herrmy86/Voxel-SLAM-intensity/tree/main
ToDo Build issues:
- https://github.com/xuankuzcr/Global-LVBA (2025, no publication)
- https://github.com/sjtuyinjie/Ground-Fusion2 (2025, IROS, BIB)
- https://github.com/ethz-asl/COIN-LIO (2024, ICRA, BIB)
- https://github.com/deepuav/AdaTrajLo (2024, RA-L, BIB)
- https://github.com/StephLin/LIO-SEGMOT (2023, ICRA, BIB)
- https://github.com/XikunLiu-huskit/GLIO (2023, TIV, BIB)
- https://github.com/tiev-tongji/LOG-LIO (2023, RA-L, BIB)
- https://github.com/chengwei0427/ct-lio (2023, no publication)
- https://github.com/minwoo0611/MA-LIO (2023, RA-L, BIB)
- https://github.com/chengwei0427/hm-lio (2023, no publication)
- https://github.com/RoboFeng/RI-LIO (2023, RA-L, BIB)
- https://github.com/ChaoqinRobotics/LINS---LiDAR-inertial-SLAM (2020, ICRA, BIB)
- https://github.com/HKUST-Aerial-Robotics/A-LOAM (2019, no publication)
- https://github.com/PRBonn/semantic_suma/ (2019, IEEE/RSJ, BIB)
- https://github.com/gisbi-kim/SC-LeGO-LOAM (2018, IEEE/RSJ, BIB)
ToDo: check if addressed
- https://github.com/dongjae0107/LAPS (2026)
- https://xiaofan4122.github.io/Elevator_LIO_Page/ (2026)
- CMLGF-LIO https://isprs-archives.copernicus.org/articles/XLIX-B1-2026/71/2026/ (2026)
- https://github.com/eugeniu1994/MAP_LIO (2026)
- ieeexplore.ieee.org/document/11264316/ (2026)
- https://arxiv.org/html/2608.14996v1 (2026)
- LF-GICP: Parameter-Free Degeneracy-Aware LiDAR Odometry via a Voxel-Normal Localizability Field (2026)
- https://www.mdpi.com/1424-8220/26/6/1810 (2026)
- RCP-LO: A Relative Coordinate Prediction Framework for Generalizable Deep LiDAR Odometry (2026)
- https://github.com/MIT-SPARK/spark-fast-lio (2025)
- https://arxiv.org/pdf/2503.12660 - KISS-SLAM (2025)
- https://github.com/HxCa1/BEV-LIO-LC (2025)
- https://arxiv.org/pdf/2410.05433 - 2FAST-LAMA (2025)
- https://www.mdpi.com/2072-4292/17/15/2656 (2025)
- https://arxiv.org/pdf/2410.08935 - VOXEL-SLAM (2024)
- https://github.com/hku-mars/fast-livo2 (2024)
- https://research.buaa.edu.cn/en/publications/ve-liom-a-versatile-and-efficient-lidar-inertial-odometry-and-map/ (2024)
- https://www.mdpi.com/2072-4292/15/20/5074 (2023)
- PVE-LIOM https://ieeexplore.ieee.org/document/10237251 (2023)
- https://github.com/APRIL-ZJU/Coco-LIC (2023)
- https://github.com/HViktorTsoi/PV-LIO (2023, addressed)
- https://arxiv.org/pdf/2204.04932 - SC-F-LOAM (2022)
- https://github.com/hku-mars/fast-livo (2022)
- https://www.mdpi.com/1424-8220/22/2/520 - LEGO-LOAM-SC (2022)
- https://github.com/lovelyyoshino/FAST_LIO2_Noted (2022)
- S-FAST_LIO https://github.com/zlwang7/S-FAST_LIO (2022)
- https://github.com/TixiaoShan/LVI-SAM (2021)
- https://github.com/HesaiTechnology-Spatial-Perception/FAST_LIO_Hesai (2021)
- https://arxiv.org/pdf/1907.02233 - LINS-SLAM (2020)
- https://github.com/rsasaki0109/li_slam_ros2 (2020)