Ego-Exo4D IMU retrieval

[Peipi98]

Hello!

I'm working on IMU data, and I'm interested on accelerometer and gyroscope. I've downloaded takes_noimagestreams first and then take_trajectory (following this issue)

I've noticed that the second split contains ['device_linear_velocity_x_device', 'device_linear_velocity_y_device', 'device_linear_velocity_z_device'], but they're not accelerations.

Last, I haven't fully understood how to get the IMU data aligned with "task_start_sec" and "task_end_sec". Given the following take, how can I retrieve the exact segment from here?

{'root_dir': 'takes/cmu_bike01_2',
 'take_name': 'cmu_bike01_2',
 'participant_uid': 657,
 'is_dropped': False,
 'objects': [],
 'task_id': 4001,
 'task_name': 'Remove a Wheel',
 'parent_task_id': 4000,
 'parent_task_name': 'Bike Repair',
 'take_idx': 2,
 'duration_sec': 110.93333333333334,
 'best_exo': 'cam04',
 'task_start_sec': 0.28164,
 'task_end_sec': 110.06668,
...

From the .vrs file I've got:

• IMU
  • start_sec = 284.324
  • end_sec = 373.796

Please note: those are timestamps converted from ns to s according to Ego-Exo4D docs.

Thank you in advance!

[lixinghe1999]

https://facebookresearch.github.io/projectaria_tools/docs/data_utilities/advanced_code_snippets/plotting_sensor_data

[Peipi98]

Hi @lixinghe1999 , thanks for your reply!

I've already extracted them from .vrs files by leveraging an example notebook in this repository, but I'd like to understand:

• differences between raw imu data from .vrs files and trajectory ones (even though it's quite clear since in the latter there are "velocity" columns instead of accelerations, it's just to have a confirm)
• if I have to interpret start_sec from each take as an "offset" to be summed to the .vrs timestamps

[lixinghe1999]

@Peipi98

1. As far as I know, the data from VRS file is raw enough, the unit for ACC and Gyro are m/s2 and rad/s2, respectively.
2. My solution is first to convert the imu from VRS to a numpy file; I believe the processing of VRS is pretty slow.

[Peipi98]

Yep as I said I've already done the extraction parts, but about the second point the timestamps don't start from 0 for each take.

If you look at takes example that I've previously reported, task_start_sec is different from start_sec taken from the relative .vrs file. Since each take is a part of a capture, I suppose that the timestamps are from the relative capture, and I'd like to know if I have to interpret "task_start_sec" as an offset.

[lixinghe1999]

Can you specify how to get the start_sec and end_sec of IMU in VRS file?
Such as

data_first = provider.get_imu_data_by_index(stream_id_imu, 0)
time_shift = data_last.capture_timestamp_ns

Previously I already found the IMU duration can be shorted than Video (VRS), but I do forget the potential offset

[Peipi98]

I've simply taken the first and last timestamps of the stream in the .vrs extracted IMU samples and converted them in seconds