Subtract voxelgrids?

[Ademord]

TLDR: I am trying to find a way to know "which points are new" in my second camera shot.

My proposed pipeline is depth image > point cloud > voxel grid

Then if I make two camera shots from two different angles I would get two voxel grids.

• how do I set that the generated point clouds exist in the same translation ? Do I somehow pass some info from the first one to the second one?
• How could I get: newVoxels = vGrid2 - vGrid1 ? I tried to find a subtraction operation but I could not find it in open3D.

Can someone pls help me in detail how to do these two things? :(

[ssheorey]

Hi @Ademord you will need the extrinsic matrix (rotation+translation) between the two camera poses. You can use RGBD odometry to calculate that, as long as the two camera poses are very similar. If they are quite different, you will need to do camera calibration (i.e. using a checkerboard or another target).

With that, you can transform the 2nd point cloud to the reference frame of the first point cloud. To find "new" points, you would need to check if each new voxel for presence in the old voxel grid.

[Ademord]

Thank you so much !!

I have just thought about a way to simplify my pipeline, which is voxelize all the meshes in my unity environment (would look like a minecraft scene) and just disable the gameObjects that are touched by the raycast the RL Agent has incorporated, that way every object can be "scanned" and give a reward to the agent.

and then i will also explain the procedure you explained here so that a PhD can maybe continue the research line and do these things outside of unity and maybe in some robotic environment with RL. Thanks so much!!