feat(ds): Add Boreas and BoreasRT dataset support#67
Merged
Conversation
Add support for the Boreas (outdoor autonomous driving) and BoreasRT datasets from the University of Toronto, downloadable from the public S3 bucket s3://boreas via boto3. Implements lidar (.bin float32), IMU (Applanix CSV for Boreas, Silicon Sensing DMU41 for BoreasRT), and ground truth (Euler-angle poses converted to SE3) loaders.
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adds support for two datasets from the Boreas autonomous driving suite (University of Toronto): Boreas (44 sequences with Applanix IMU) and BoreasRt (15 sequences with Silicon Sensing DMU41 IMU). Both use a Velodyne VLS-128 lidar and download data from a public S3 bucket via boto3.
Changes:
- New
Boreasdataset class with shared helpers for folder name conversion, Euler-to-rotation conversion, ground truth loading, and.binpoint cloud loading - New
BoreasRtdataset class that reuses the shared helpers but has different IMU parameters and calibration - Adds
boto3>=1.26.0as a project dependency
Reviewed changes
Copilot reviewed 4 out of 5 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
python/evalio/datasets/boreas.py |
New Boreas dataset class with 44 sequences and shared helper functions |
python/evalio/datasets/boreas_rt.py |
New BoreasRt dataset class with 15 sequences and DMU41 IMU config |
python/evalio/datasets/__init__.py |
Registers both new dataset classes |
pyproject.toml |
Adds boto3>=1.26.0 dependency |
uv.lock |
Lock file updated with boto3 and transitive dependencies |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
You can also share your feedback on Copilot code review. Take the survey.
python/evalio/datasets/boreas_rt.py
Outdated
Comment on lines
+7
to
+11
| from evalio._cpp.types import Duration, LidarMeasurement, Point, Stamp # type: ignore | ||
| from evalio.datasets.loaders import RawDataIter | ||
| from evalio.types import ( | ||
| SE3, | ||
| SO3, |
python/evalio/datasets/boreas.py
Outdated
Comment on lines
+53
to
+54
| File format: t, x, y, z, vx, vy, vz, r, p, y, wz, wy, wx | ||
| where t is in microseconds and r, p, y are roll, pitch, yaw in radians. |
Owner
Author
|
Welp, gave Claude a shot at solving this and it failed spectacularly lol. After much cleaning up, this is finally all working! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds support for two datasets from the Boreas autonomous driving dataset suite (University of Toronto):
Boreas— outdoor driving sequences with a Velodyne VLS-128 lidar and Applanix IMUBoreasRt— newer sequences with the same lidar but a Silicon Sensing DMU41 IMUBoth datasets are downloaded from the public S3 bucket
s3://boreas(no credentials required) usingboto3.What's included
python/evalio/datasets/boreas.py—Boreasdataset class plus shared helpers:_seq_name_to_folder— converts evalio snake_case seq names to S3 folder names_yaw_pitch_roll_to_rot— ZYX Euler → rotation matrix_load_boreas_gt— parsesapplanix/lidar_poses.csv(timestamps in µs, RPY angles) intoTrajectory_load_boreas_bin— loads VLS-128.binpoint clouds (float32[x,y,z,intensity,ring,t])python/evalio/datasets/boreas_rt.py—BoreasRtdataset class, re-uses the shared helpers abovepython/evalio/datasets/__init__.py— registers both new dataset classespyproject.toml/uv.lock— addsboto3>=1.26.0dependencyCalibration sources
All hardcoded extrinsics and noise parameters include source-link comments pointing to the actual calibration files on S3 or the dataset's sensor spec sheet.
T_applanix_lidar(Boreas)s3://boreas/boreas-2020-11-26-13-58/calib/T_applanix_lidar.txtT_applanix_lidar(BoreasRT)s3://boreas/boreas-2024-12-03-12-54/calib/T_applanix_lidar.txtT_applanix_dmu(BoreasRT)s3://boreas/boreas-2024-12-03-12-54/calib/T_applanix_dmu.txtTesting