Trajectory Metadata & Config Parsing

CITATION.bib

@@ -0,0 +1,9 @@
+@misc{potokar2025_evaluation_lidar_odometry,
+  title         = {A Comprehensive Evaluation of LiDAR Odometry Techniques},
+  author        = {Easton Potokar and Michael Kaess},
+  year          = {2025},
+  eprint        = {2507.16000},
+  archiveprefix = {arXiv},
+  primaryclass  = {cs.RO},
+  url           = {https://arxiv.org/abs/2507.16000}
+}

README.md

@@ -13,15 +13,15 @@ Specifically, it provides a common interface for connecting LIO datasets and LIO
 
 ## Installation
 
-evalio is available on PyPi, so simply install via your favorite python package manager,
+evalio is available on PyPi (with all pipelines compiled in!), so simply install via your favorite python package manager,
 ```bash
 uv add evalio      # uv
 pip install evalio # pip
 ```
 
-## Usage
+## Basic Usage
 
-evalio can be used both as a python library and as a CLI for both datasets and pipelines.
+evalio can be used both as a python library and as a CLI for both datasets and pipelines. We cover just the tip of the iceberg here, so please check out the [docs](https://contagon.github.io/evalio/) for more information.
 
 ### Datasets
 
@@ -30,67 +30,24 @@ Once evalio is installed, datasets can be listed and downloaded via the CLI inte
 evalio ls datasets
 evalio download hilti_2022/basement_2
 ```
-evalio downloads data to the `EVALIO_DATA` environment variable, or if unset to the local folder `./evalio_data`. All the trajectories in a dataset can also be downloaded by using the wildcard `hilti_2022/*`, making sure to escape the asterisk as needed.
-
-> [!TIP]
-> evalio also comes with autocomplete, which makes typing the long dataset and pipeline names much easier. To install, do one of the following,
-> ```bash
-> eval "$(evalio --show-completion)" # install for the current session
-> evalio --install-completion        # install for all future sessions
-
-> [!NOTE]
-> Many datasets use [gdown](https://github.com/wkentaro/gdown) to download datasets from google drive. Unfortunately, this can occasionally be finicky due to google's download limits, however [downloading cookies from your browser](https://github.com/wkentaro/gdown?tab=readme-ov-file#i-set-the-permission-anyone-with-link-but-i-still-cant-download) can often help.
-
 
 Once downloaded, a trajectory can then be easily used in python,
 ```python
-from evalio.datasets import Hilti2022
+from evalio import datasets as ds
 
 # for all data
-for mm in Hilti2022.basement_2:
+for mm in ds.Hilti2022.basement_2:
     print(mm)
 
 # for lidars
-for scan in Hilti2022.basement_2.lidar():
+for scan in ds.Hilti2022.basement_2.lidar():
     print(scan)
 
 # for imu
-for imu in Hilti2022.basement_2.imu():
+for imu in ds.Hilti2022.basement_2.imu():
     print(imu)
 ```
 
-For example, you can easily get a single scan to plot a bird-eye view,
-```python
-import matplotlib.pyplot as plt
-import numpy as np
-
-# get the 10th scan
-scan = Hilti2022.basement_2.get_one_lidar(10)
-# always in row-major order, with stamp at start of scan
-x = np.array([p.x for p in scan.points])
-y = np.array([p.y for p in scan.points])
-z = np.array([p.z for p in scan.points])
-plt.scatter(x, y, c=z, s=1)
-plt.axis('equal')
-plt.show()
-```
-evalio also comes with a built wrapper for converting to [rerun](rerun.io) types,
-```python
-import rerun as rr
-from evalio.rerun import convert
-
-rr.init("evalio")
-rr.connect_tcp()
-for scan in Hilti2022.basement_2.lidar():
-    rr.set_time("timeline", timestamp=scan.stamp.to_sec())
-    rr.log("lidar", convert(scan, color=[255, 0, 255]))
-```
-
-> [!NOTE]  
-> To run the rerun visualization, rerun must be installed. This can be done by installing `rerun-sdk` or `evalio[vis]` from PyPi.
-
-We recommend checking out the [base dataset class](python/evalio/datasets/base.py) for more information on how to interact with datasets.
-
 ### Pipelines
 
 The other half of evalio is the pipelines that can be run on various datasets. All pipelines and their parameters can be shown via,
@@ -105,15 +62,13 @@ This will run the pipeline on the dataset and save the results to the `results`
 ```bash
 evalio stats results
 ```
-> [!NOTE]  
-> KissICP does poorly by default on hilti_2022/basement_2, due to the close range and large default voxel size. You can visualize this by adding `-vvv` to the `run` command to visualize the trajectory in rerun.
 
 More complex experiments can be run, including varying pipeline parameters, via specifying a config file,
 ```yaml
 output_dir: ./results/
 
 datasets:
-  # Run on all of newer college trajectories
+  # Run on all of hilti trajectories
   - hilti_2022/*
   # Run on first 1000 scans of multi campus
   - name: multi_campus/ntu_day_01
@@ -126,7 +81,7 @@ pipelines:
   - name: kiss_tweaked
     pipeline: kiss
     deskew: true
-    # Some of these datasets need smaller voxel sizes
+    # Sweep over voxel size parameter
     sweep:
       voxel_size: [0.1, 0.5, 1.0]
 
@@ -135,43 +90,22 @@ This can then be run via
 ```bash
 evalio run -c config.yml
 ```
-That's about the gist of it! Try playing around the CLI interface to see what else is possible, such as a number of visualization options using rerun. Feel free to open an issue if you have any questions, suggestions, or problems.
-
-## Custom Datasets & Pipelines
-We understand that using an internal or work-in-progress datasets and pipelines will often be needed, thus evalio has full support for this. As mentioned above, we recommend checking out our [example](https://github.com/contagon/evalio-example) for more information how to to do this (it's pretty easy!). 
-
-The TL;DR version, a custom dataset can be made via inheriting from the `Dataset` class in python only, and a custom pipeline from inheriting the `Pipeline` class in either C++ or python. These can then be made available to evalio via the `EVALIO_CUSTOM` env variable point to the python module that contains them.
-
-We **highly** recommend making a PR to merge your custom datasets or pipelines into evalio once they are ready. This will make it more likely the community will use and cite your work, as well as increase the usefulness of evalio for everyone.
-
-## Building from Source
-
-While we recommend simply installing the python package using your preferred python package manager (our is `uv`), we've attempted to make building from source as easy as possible. We generally build through [scikit-core-build](https://scikit-build-core.readthedocs.io/) which provides a simple wrapper for building CMake projects as python packages. `uv` is our frontend of choice for this process, but it is also possible via pip
-```bash
-uv sync          # uv version
-pip install -e . # pip version
-```
-
-Of course, building via the usual `CMake` way is also possible, with the only default dependency being `Eigen3`,
-```bash
-mkdir build
-cd build
-cmake ..
-make
-```
-
-By default, all pipelines are not included due to their large dependencies. CMake will look for them in the `cpp/bindings/pipelines-src` directory. If you'd like to add them, simply run the `clone_pipelines.sh` script that will clone and patch them appropriately. 
-
-When these pipelines are included, the number of dependencies increases significantly, so have provided a [docker image](https://github.com/contagon/evalio/pkgs/container/evalio_manylinux_2_28_x86_64) that includes all dependencies for building as well as a VSCode devcontainer configuration. When opening in VSCode, you'll automatically be prompted to open in this container.
 
 ## Contributing
 
-Contributions are always welcome! Feel free to open an issue, pull request, etc. We're happy to help you get started. The following are rough instructions for specifically adding additional datasets or pipelines.
-
-### Datasets
-Datasets are easy to add, simply drop your file into the [python/evalio/datasets](python/evalio/datasets/) folder, and add it into the [init](python/evalio/datasets/__init__.py) file.
-
-### Pipelines
-If adding in a python pipeline, it's near identical to adding a dataset. Drop your file into the [python/evalio/pipelines](python/evalio/pipelines/) folder, and add it into the [init](python/evalio/pipelines/__init__.py) file.
-
-C++ pipelines are more involved (and probably worth the effort). Your header file belongs in the [cpp/bindings/pipelines](cpp/bindings/pipelines/) folder. To get it to build, make sure it's added to [clone_pipelines.sh](clone_pipelines.sh), the proper [CMakeLists.txt](cpp/bindings/CMakeLists.txt), and the [bindings.h] header. Finally, make sure all dependencies are also added to the docker build script, found in the [docker](docker/) folder.
+Contributions are always welcome! Feel free to open an issue, pull request, etc. The documentation has a more details on developing new datasets and pipelines.
+
+## Citation
+
+If you use evalio in your research, please cite the following paper,
+```bibtex
+@misc{potokar2025_evaluation_lidar_odometry,
+      title={A Comprehensive Evaluation of LiDAR Odometry Techniques}, 
+      author={Easton Potokar and Michael Kaess},
+      year={2025},
+      eprint={2507.16000},
+      archivePrefix={arXiv},
+      primaryClass={cs.RO},
+      url={https://arxiv.org/abs/2507.16000}, 
+}
+```

cpp/bindings/main.cpp

@@ -8,6 +8,8 @@
 namespace nb = nanobind;
 
 NB_MODULE(_cpp, m) {
+  nb::set_leak_warnings(false);
+
   m.def(
     "abi_tag",
     []() { return nb::detail::abi_tag(); },

cpp/bindings/ros_pc2.h

@@ -8,6 +8,7 @@
 #include <cmath>
 #include <cstddef>
 #include <fstream>
+#include <map>
 
 #include "evalio/types.h"
 
@@ -386,6 +387,86 @@ inline LidarMeasurement helipr_bin_to_evalio(
   return mm;
 }
 
+/// Parse a CSV line into an SE3 object. The idx map should contain the indices
+/// of the required fields: "qw", "qx", "qy", "qz", "x", "y", "z".
+inline std::pair<Stamp, SE3> parse_csv_line(
+  const std::string& s,
+  const char delimiter,
+  const std::map<std::string, int>& idx
+) {
+  std::stringstream ss(s);
+  std::string item;
+  std::vector<std::string> elems;
+  while (std::getline(ss, item, delimiter)) {
+    elems.push_back(item);
+  }
+
+  // Parse out the fields
+  SO3 r = SO3 {
+    .qx = std::stod(elems[idx.at("qx")]),
+    .qy = std::stod(elems[idx.at("qy")]),
+    .qz = std::stod(elems[idx.at("qz")]),
+    .qw = std::stod(elems[idx.at("qw")]),
+  };
+  Eigen::Vector3d t = Eigen::Vector3d(
+    std::stod(elems[idx.at("x")]),
+    std::stod(elems[idx.at("y")]),
+    std::stod(elems[idx.at("z")])
+  );
+
+  Stamp stamp;
+  // If both sec/nsec are given
+  if (idx.count("sec") && idx.count("nsec")) {
+    stamp = Stamp {
+      .sec = static_cast<uint32_t>(std::stoul(elems[idx.at("sec")])),
+      .nsec = static_cast<uint32_t>(std::stoul(elems[idx.at("nsec")]))
+    };
+  }
+
+  // If only sec is given, split it into sec/nsec
+  else if (idx.count("sec")) {
+    // Find decimal place
+    std::string sec_str = elems[idx.at("sec")];
+    size_t dot_pos = sec_str.find('.');
+    if (dot_pos == std::string::npos) {
+      throw std::runtime_error("Failed to find decimal in sec field.");
+    }
+
+    // extract sec
+    uint32_t sec_part = std::stoul(sec_str.substr(0, dot_pos));
+
+    // extract & pad nsec
+    std::string nsec_str = sec_str.substr(dot_pos + 1);
+    if (nsec_str.size() > 9) {
+      throw std::runtime_error("Too many digits in fractional part of sec.");
+    } else if (nsec_str.size() < 9) {
+      nsec_str += std::string(9 - nsec_str.size(), '0');
+    }
+    uint32_t nsec_part = std::stoul(nsec_str);
+
+    stamp = Stamp {.sec = sec_part, .nsec = nsec_part};
+  }
+
+  // If only nsec is given
+  else if (idx.count("nsec")) {
+    stamp = Stamp::from_nsec(std::stoul(elems[idx.at("nsec")]));
+  }
+
+  // If neither is given, throw an error
+  else {
+    throw std::runtime_error("Must have at least one of 'sec' or 'nsec'.");
+  }
+
+  return std::make_pair(stamp, SE3(r, t));
+}
+
+// Returns False if a is closer to idx, True if b is closer to idx
+inline bool closest(const Stamp& idx, const Stamp& a, const Stamp& b) {
+  auto a_diff = std::abs((a - idx).to_nsec());
+  auto b_diff = std::abs((b - idx).to_nsec());
+  return a_diff > b_diff;
+}
+
 // ---------------------- Create python bindings ---------------------- //
 inline void makeConversions(nb::module_& m) {
   nb::enum_<DataType>(m, "DataType")
@@ -454,6 +535,9 @@ inline void makeConversions(nb::module_& m) {
   m.def("helipr_bin_to_evalio", &helipr_bin_to_evalio);
   // botanic garden velodyne reordering
   m.def("fill_col_split_row_velodyne", &fill_col_split_row_velodyne);
+
+  m.def("parse_csv_line", &parse_csv_line);
+  m.def("closest", &closest);
 }
 
 } // namespace evalio