MA-VIED is a real-world extensive set of data to support the development of solutions and algorithms for sensor fusion, event-based and frame-based visual tracking, including visual, visual-inertial odometry and SLAM applications.
The MA-VIED Dataset is intended for
- evaluating the performance of vision algorithms in the context of Visual Inertial Odometry (VIO) and Simultaneous Localization and Mapping (SLAM) tasks.
- supporting research endeavors that leverage large-scale driving datasets, especially those equipped with novel neuromorphic vision sensors.
MA-VIED is released under the following License:
Attribution-NonCommercial-ShareAlike 3.0 CC BY-NC-SA 3.0.
This means it is possible:
- to copy, distribute, display, and perform the work.
- to make derivative works.
Under the following conditions:
- Attribution: You must give the original author credit.
- Non-Commercial — You may not use this work for commercial purposes.
- Share Alike — If you alter, transform, or build upon this work, you may distribute the resulting work only under a licence identical to this one.
Notice: This automotive dataset is provided as-is and may contain personal information. We take user privacy seriously and have implemented measures to minimize the inclusion of sensitive data. However, given the complexity of real-world data, it is possible that some personal information may be present.
User Notification and Data Removal:
Users are encouraged to review the dataset carefully. If you identify any personal data that should not be included, please promptly raise an issue on our GitHub repository. We are committed to promptly addressing and resolving privacy concerns. Upon receiving a valid request, we will make reasonable efforts to remove or anonymize the specified personal data from the dataset.
Important Considerations:
The dataset is intended for research and educational purposes. Users are responsible for complying with privacy and data protection regulations when using this dataset. We do not assume liability for any unauthorized use of the dataset.
Contact Information:
For privacy-related concerns or data removal requests, please contact us at giuseppe.mollica1@studenti.unipg.it
If you use MA-VIED in an academic work, please cite:
@article{10254473,
author={Mollica, Giuseppe and Felicioni, Simone and Legittimo, Marco and Meli, Leonardo and Costante, Gabriele and Valigi, Paolo},
journal={IEEE Transactions on Intelligent Transportation Systems},
title={MA-VIED: A Multisensor Automotive Visual Inertial Event Dataset},
year={2024},
volume={25},
number={1},
pages={214-224},
keywords={Cameras;Sensors;Wheels;Sensor fusion;Odometry;Standards;Visualization;Visual inertial odometry;event vision;MA-VIED automotive dataset;sensor fusion},
doi={10.1109/TITS.2023.3312355}
}
We compiled a dataset using a sensor setup installed on the roof of an electric KIA Soul. Below, we outline the vehicle specifications and the sensor equipment employed. The setup includes Two Wheel Pulse Transducers (WPT) connected to both the left and right wheels of the vehicle. Additionally, the inertial-vision (IV) system comprises an event camera, a standard camera, and an IMU. The in-car sensor data can be accessed through the OBDII interface.
 |
 |
|---|
| Vehicle Measurements |
|---|
 |
 |
 |
 |
|---|---|
| Position and Inertial Measurement units (GNSS/INS): MTi-G-710 GNSS/INS. | |
| Position measurement units (RTK GPS): Piksi Multi GNSS Module | |
| Frame Camera Module: daA1920-160um (CS-Mount) - Basler dart. | |
| -> Lens: Basler Lens C125-0618-5M-P f6mm. | |
| -> Polarizer Filter: Linear Polarizer Filter. | |
| Event Camera Module: Prophesee PPS3MVC. | |
| -> Lens: SV-0813V. | |
| Odometry sensors: Kistler WPT sensors . | |
| Microcontroller Board Teensy® 4.0 Development Board. | |
| Main computer: The system is powered by an Intel Core i7-10710U Processor with integrated Intel Graphics, housed in an Intel NUC 10 Performance Kit (NUC10i7FNK) mini PC. To optimize performance and storage capacity, we opted for two distinct storage solutions. The first is a PCI Express x4 NVMe 1.3, 256GB drive dedicated to the operating system (Ubuntu 20.04 LTS 64-bit). The second is a SATA SSD 2.5", 1TB disk allocated for data storage | |
All sensor data is synchronized using a trigger signal dispatched by a microcontroller, ensuring precise timestamps for accurate temporal alignment.
The microcontroller computes precise millisecond timestamps for each IMU measurement at a frequency of 200Hz. At specific timestamps (28Hz), it initiates various actions:
- Triggers the camera (via the trigger line) to capture a new image.
- Instructs the event camera to record the trigger-in as event data at a rate of 100Hz.
- Commands both RTK receivers (at 28Hz) to log the input trigger.
Timestamps and trigger sequence numbers are available through three distinct ROS topics. Given the varied requirements for alignment, we chose to retain the triggering information, allowing users the flexibility to implement post-processing and alignment procedures as needed.
The 3D CAD model for the roof-mounted setup can be found in the 'additional_material' folder on this project's page.
Note: Due to the unavailability of trigger input, the CAN and wheel pulse transducer data are not hardware synchronized with other sensors.
Note: Prophesee EVKs and sensors support external triggering (input or output signals), enabling control of the sensor and data synchronization across multiple devices. For details on the event camera triggering mechanism, please refer to the Prophesee documentation.
MA-VIED was collected using a Kia Soul electric vehicle equipped with diverse heterogeneous sensors, capturing crucial information for vehicle localization and tracking. The dataset comprises sequences recorded during 13 experimental sessions in varied driving environments (highway, country road, city, racetrack-like) and styles (smooth and rapid maneuvers). These sessions took place in December 2021, as the electric vehicle navigated for several kilometers through different streets in Passignano sul Trasimeno (PG, Umbria, Italy).
The accompanying table illustrates the percentage of RTK fixed data in relation to the total GPS data for both RTK antennas.
| Sequence name | Length(m) | Duration(sec) | Avg. vel. (km/h) | Max. vel. (km/h) | RTK Data Percentage (RTK0 - RTK1) | Avg. Event Per Sec. | Date: dd, mm, yyyy |
|---|---|---|---|---|---|---|---|
| Parking_maneuver_slow | 79.49 | 120 | 0.66 | 5.29 | 100 - 100 | 800663 | 12, December, 2021 |
| Parking_maneuver_medium | 150.67 | 106 | 2.69 | 11.31 | 100 - 100 | 2519114 | 12, December, 2021 |
| Parking_maneuver_fast | 164.31 | 81 | 2.63 | 15.98 | 100 - 100 | 2514363 | 12, December, 2021 |
| Countryside | 2458.83 | 309 | 28.44 | 46.32 | 97.06 - 95.35 | 724782 | 12, December, 2021 |
| Loop_city_slow | 1021.68 | 214 | 17.19 | 30.70 | 100 - 98.23 | 1067229 | 12, December, 2021 |
| Loop_city_medium | 1010.69 | 129 | 19.24 | 29.17 | 100 - 100 | 1564171 | 12, December, 2021 |
| Loop_city_fast | 525.91 | 97 | 28.72 | 57.45 | 100 - 100 | 2474209 | 12, December, 2021 |
| Highway | 5308.30 | 264 | 67.83 | 105.93 | 97.51 - 100 | 641154 | 12, December, 2021 |
| City_parking_maneuver | 83.26 | 104 | 0.47 | 10.59 | 100 - 100 | 469747 | 12, December, 2021 |
| City_1 | 1705.83 | 239 | 25.12 | 54.67 | 97.4 - 96.88 | 1227432 | 12, December, 2021 |
| City_2 | 494.75 | 100 | 17.89 | 37.25 | 100 - 100 | 733366 | 12, December, 2021 |
| City_3 | 1527.4938 | 135 | 39.66 | 59.70 | 100 - 99.44 | 1757991 | 12, December, 2021 |
| City_4 | 1394.66 | 112 | 44.03 | 58.17 | 88.8 - 87.39 | 2541769 | 12, December, 2021 |
Note: The RTK GPS provides data in three modes, ranked by increasing accuracy: Autonomous, RTK Float, and RTK Fix. The RTK Fix mode is the most precise, with a deviation of 1-6cm. However, it is important to note that the high sensitivity of the RTK link makes it susceptible to signal loss in the presence of elements such as trees or buildings, which can result in reduced positioning accuracy. Across various sequences, the RTK evaluation frequently oscillates between fixed and float statuses.
The released sequences have been meticulously curated to ensure high-quality data and variability, particularly in terms of lighting conditions and texturing. Additionally, the sequences encompass diverse use cases, including event-based autonomous parking, race tracking for virtual coach applications, and various robotic challenges such as Visual Odometry (VO), Visual-Inertial Odometry (VIO), and Simultaneous Localization and Mapping (SLAM).
In addition to the evaluation sequences, we are providing our setup CAD model and calibration sequences. This enables users to conduct their calibration, even though we also provide the calibration results.
LINK TO DATASET DOWNLOAD
http://sira.diei.unipg.it/supplementary/public/Datasets/MA-VIED/Data/
We used ROS Melodic, utilizing the ROSBAG structure and the broader ROS infrastructure to collect and manage all data.
The rosbag package facilitates the recording of various topics and messages into a single file, enabling batch execution for experiment reproduction. The subsequent table provides detailed information about the collected data, including message types and available topics within this dataset.
| Topic | Message Type | Description |
|---|---|---|
| /KIA_SOUL_EV/ART/car_state | roscco_art/Car_State | Parsed data from vehicle CAN bus (for additional information, refer to section 3.1.1.). |
| /can_frame_can0 | can_msgs/Frame | RAW data from vehicle CAN bus. |
| /arduino/timeref_events | sensor_msgs/TimeReference | Microcontroller internal time reference of trigger signal sent to Prophesee event camera. (refer to synchronization scheme in the syncronization section). |
| /arduino/timeref_mono | sensor_msgs/TimeReference | Microcontroller internal time reference of trigger signal sent to Basler camera. (refer to synchronization scheme in the syncronization section). |
| /arduino/timeref_rtk | sensor_msgs/TimeReference | Microcontroller internal time reference of trigger signal sent to the two Piksi Multi RTK receiver. (refer to synchronization scheme in the syncronization section). |
| /imu/acceleration | geometry_msgs/Vector3Stamped | Acceleration about (𝑥, 𝑦, 𝑧) axes from MTi-G-710 IMU. |
| /imu/angular_velocity | geometry_msgs/Vector3Stamped | Angular velocity about (𝑥, 𝑦, 𝑧) axes from MTi-G-710 IMU. |
| /imu/data | sensor_msgs/Imu | Angular velocity and acceleration about (𝑥, 𝑦, 𝑧) axes from MTi-G-710 IMU. |
| /imu/mag | geometry_msgs/Vector3Stamped | Magnetometer data from MTi-G-710 IMU. |
| /imu/time_ref | sensor_msgs/TimeReference | MTi-G-710 IMU Sensor time reference starting from device power on. |
| /gnss | sensor_msgs/NavSatFix | GPS GNSS data from XSens MTi-G-710 IMU. |
| /piksi_ttyUSB/piksi_multi_base_station/enu_point_fix/piksi_ttyUSB/piksi_multi_base_station/enu_point_float/piksi_ttyUSB/piksi_multi_base_station/enu_point_spp | geometry_msgs/PointStamped | Piksi localization data in geometry_msgs/PointStamped format (by accuracy, from grater to lower: fix,float and spp). |
| /piksi_ttyUSB/piksi_multi_base_station/enu_pose_best_fix/piksi_ttyUSB/piksi_multi_base_station/enu_pose_fix/piksi_ttyUSB/piksi_multi_base_station/enu_pose_float/piksi_ttyUSB/piksi_multi_base_station/enu_pose_spp | geometry_msgs/PoseWithCovarianceStamped | Piksi localization data in geometry_msgs/PoseWithCovarianceStamped format (by accuracy, from grater to lower: fix,float and spp). Best fix contains the available data with greater accuracy. |
| /piksi_ttyUSB/piksi_multi_base_station/enu_transform_fix/piksi_ttyUSB/piksi_multi_base_station/enu_transform_float/piksi_ttyUSB/piksi_multi_base_station/enu_transform_spp | geometry_msgs/TransformStamped | Piksi localization data in geometry_msgs/TransformStamped format (by accuracy, from grater to lower: fix,float and spp). |
| /piksi_ttyUSB/piksi_multi_base_station/navsatfix_best_fix/piksi_ttyUSB/piksi_multi_base_station/navsatfix_fix/piksi_ttyUSB/piksi_multi_base_station/navsatfix_float/piksi_ttyUSB/piksi_multi_base_station/navsatfix_spp | sensor_msgs/NavSatFix | Piksi localization data in sensor_msgs/NavSatFix format (by accuracy, from grater to lower: fix,float and spp). Best fix contains the available data with greater accuracy. |
| /piksi_ttyUSB/piksi_multi_base_station/ext_event | piksi_rtk_msgs/ExtEvent | Piksi external trigger event information. |
| /piksi_ttyUSB/piksi_multi_base_station/gps_time/piksi_ttyUSB1/piksi_multi_base_station/utc_time | piksi_rtk_msgs/GpsTimeMulti | Piksi GPS and UTC data time. |
| /piksi_ttyUSB/piksi_multi_base_station/imu | piksi_rtk_msgs/GpsTimeMulti | Angular velocity and acceleration about (𝑥, 𝑦, 𝑧) axes from Piksi multi. |
| /piksi_ttyUSB/piksi_multi_base_station/mag | piksi_rtk_msgs/GpsTimeMulti | Magnetometer data from Piksi multi. |
| /piksi_ttyUSB/piksi_multi_base_station/mag | piksi_rtk_msgs/MeasurementState_V2_4_1 | Piksi measurement accuracy information. |
| /piksi_ttyUSB/piksi_multi_base_station/vel_ned | piksi_rtk_msgs/VelNed | Velocity Solution in NED (North-East-Down) coordinates. |
| /pylon_camera_node/currentParams | camera_control_msgs/currentParams | Internal camera configuration. |
| /pylon_camera_node/image_raw | sensor_msgs/Image | RAW image from Basler camera. |
| /cd_events_buffer | prophesee_event_msgs/EventArray | RAW event stream from Prophesee camera. |
| /cd_trigger_buffer | sensor_msgs/TimeReference | Triggering input information from Prophesee camera (refer to sinchronization section for in depth explanation of the synchronization and triggering mechanisms). |
| /odometry/from_wheels | nav_msgs/Odometry | Processed data from WPT wheels odometry sensor (Vehicle speed along the longitudinal axe). |
| /odometry/from_wheels | nav_msgs/Odometry | Processed data from WPT wheels odometry sensor fused with IMU Xsens G-710 data (Fused vehicle speed along the three axes). |
Note: for all piksi multi data, 0 refers to front RTK antenna and 1 refers to rear RTK antenna
The KIA_SOUL_EV/ART/car_state topic contains information related to wheel ticks (utilizing Kistler wheel encoders), steering angle (vehicle CAN), current gear position (vehicle CAN), and brake pedal pressure (expressed in the range 0-1 and extracted from vehicle CAN). Specifically:
encoder_pulses_RR: Count of ticks for the right wheel.encoder_pulses_RL: Count of ticks for the left wheel.steering_angle: Steering wheel angle expressed in degrees.actual_gear: Defines the current gear according to the following numbering:NO_COMMAND: 0NEUTRAL: 1FORWARD: 2REVERSE: 3EMERGENCY_STOP: 4
brake_pressure: Brake pedal pressure status (0 for brake completey released and 1 for brake completely pressed).
This topic encompasses the RAW unparsed data retrieved from the Can vehicle interface.
Piksi RTK custom messages. For additional information, please refer to https://github.com/ethz-asl/ethz_piksi_ros.
We provide the following calibration parameters for cameras and IMU sensors:
- Intrinsic camera matrix for event and frame cameras,
- IMU biases and noise,
- Extrinsic parameters: event camera to IMU, camera frame to IMU, and RTK to IMU parameters for both RTK antennas (derived from the 3D CAD model).
In addition to the evaluation sequences, our release includes the setup CAD model and calibration sequences. This empowers users to perform their calibration, even though we do provide the calibration results.
The following sequences are available:
| Sequence Name | Comment |
|---|---|
| basler_calib | Calibration sequence for Basler frame-based camera intrinsic parameters, using a checkerboard pattern. |
| imu_cam_calib | Calibration sequence for Basler-to-IMU and Prophesee-to-IMU extrinsic calibration. |
| imu_steady_calib | Calibration sequence for IMU intrinsic parameters. |
| prophesee_calib | Calibration sequence for Prophesee event-based camera intrinsic parameters, using a blinking checkerboard pattern. |
The checkerboards used for camera and camera-to-IMU calibrations are as follows:
| APRIL TAG Checkerboard Measurement | |
|---|---|
 |
|
| Tag Size: 3 [cm] | |
| Tag Spacing: 0.4 [cm] | |
| Columns: 6 | |
| Rows: 6 | |
| Checkerboard Measurement | |
|---|---|
 |
|
| Square Size:: 3.5 [cm] | |
| Columns: 8 | |
| Rows: 5 | |
LINK TO CALIBRATION DATASET DOWNLOAD
http://sira.diei.unipg.it/supplementary/public/Datasets/MA-VIED/Calibration/
| Imu to Basler transformation matrix | ||||
|---|---|---|---|---|
 |
||||
| 0.0430021 | 0.0113910 | 0.9990100 | -0.0013475 | |
| -0.9982296 | -0.0106354 | 0.0434319 | -0.0013475 | |
| 0.0110899 | -0.9991091 | 0.0096234 | -0.0064315 | |
| 0.0 | 0.0 | 0.0 | 1.0 | |
| Imu to Prophesee transformation matrix | ||||
|---|---|---|---|---|
 |
||||
| 0.0218143 | 0.0255480 | 0.9965680 | 0.0004503 | |
| -0.9985160 | -0.0226867 | 0.0138182 | 0.0154573 | |
| 0.0098995 | -0.9958710 | 0.0254170 | 0.0068194 | |
| 0.0 | 0.0 | 0.0 | 1.0 | |
| Parameter | Value |
|---|---|
| Image Width | 1920 |
| Image Height | 1200 |
| Focal Length (fx) | 1760.33875 |
| Focal Length (fy) | 1771.39124 |
| Principal Point (cx) | 926.72117 |
| Principal Point (cy) | 608.13289 |
| Distortion Coefficient (k1) | -0.077591 |
| Distortion Coefficient (k2) | 0.103893 |
| Distortion Coefficient (p1) | 0.000489 |
| Distortion Coefficient (p2) | -0.001018 |
| Parameter | Value |
|---|---|
| Image Width | 640 |
| Image Height | 480 |
| Focal Length (fx) | 547.65806183832740e+02 |
| Focal Length (fy) | 547.33186007966117e+02 |
| Principal Point (cx) | 324.91636718815175e+02 |
| Principal Point (cy) | 209.11828571087136e+02 |
| Distortion Coefficient (k1) | -1.7860443233929391e-01 |
| Distortion Coefficient (k2) | 9.8421739917362486e-02 |
| Distortion Coefficient (p1) | 0.0 |
| Distortion Coefficient (p2) | 0.0 |
| Parameter | Value | Comment |
|---|---|---|
| Accelerometer Noise (acc_n) | 7.8531619738053877e-03 | Accelerometer measurement noise standard deviation. |
| Gyroscope Noise (gyr_n) | 1.5641921981365443e-04 | Gyroscope measurement noise standard deviation. |
| Accelerometer Bias (acc_w) | 3.2064659601889169e-04 | Accelerometer bias random work noise standard deviation. |
| Gyroscope Bias (gyr_w) | 5.8371294942786571e-05 | Gyroscope bias random work noise standard deviation. |
| Gravity Magnitude (g_norm) | 9.750086155697764 | Gravity magnitude. |
This project is a collaborative effort between ART S.p.A and the University of Perugia, Faculty of Engineering. The collaboration brings together industry expertise and academic research to contribute to the development of this automotive dataset.
| Information | |
|---|---|
 |
ART S.p.A Website: www.artgroup-spa.com Address: Località Pischiello, 20, 06065 Passignano sul Trasimeno PG. |
 |
University of Perugia, Faculty of Engineering Website: https://ing.unipg.it/ Address: Via Goffredo Duranti, 93, 06125 Perugia PG |