Transformation between ARKit mesh and Point Cloud

[cy94]

Hello,

It looks like the 3D bounding boxes can only be used on the ARKit reconstructions.
Could you provide the transformation between this mesh and the point clouds, so that the bounding boxes can be used on the point cloud as well?
Could you also explain the difference between data['segments']['obbAligned'] and data['segments']['obb'] in the annotations file?

Thanks!

[PeterZheFu]

Hi @cy94 , thanks for your interest and questions~ Let me take a quick look and get back to you.

[PeterZheFu]

Hi @cy94 , just to clarify, for your question of
"Could you provide the transformation between this mesh and the point clouds, so that the bounding boxes can be used on the point cloud as well?", we have a sample script here https://github.com/apple/ARKitScenes/blob/main/threedod/README.md#preparing-whole-scene-offline-data-with-visualizations that draws the bounding boxes on the point clouds. Would this satisfy your requirement? Thanks.

[cy94]

@PeterZheFu thanks for the quick reply. It looks like this generates the point cloud by back-projecting RGB image using the depth values, right? i.e, its the same space as the ARKit mesh. Can we use these bounding boxes on the laser scanner point cloud as well?

[chaoqun-wang0721]

Hi @cy94 , data['segments']['obbAligned'] is GT in the world coordinate system, which should be used by users. A reference example can be found in the helper script mentioned in the link above. Thank you.

[PeterZheFu]

Hi @cy94 ,
Thanks for your interest in our dataset. For your question, "It looks like the 3D bounding boxes can only be used on the ARKit reconstructions.
Could you provide the transformation between this mesh and the point clouds, so that the bounding boxes can be used on the point cloud as well?", I am afraid that at this point we do not have plans to provide such information. I will keep you updated if we plan to release it.