About obtaining pose from point clouds

[vtasStu]

Hi Yan Di, I see the input of your network is P = points - points.mean(dim=1, keepdim=True)，where points is obtained from the backprojection of the depth map.

In my shallow knowledge, two pieces of information are needed to obtain the pose(R only discussed), the points P=R @ P_ori after the R transformation and the original points P_ori.

The input of network only has P=R @ P_ori, how does the network get rotationR without knowing the original points P_ori？

[shangbuhuan13]

In the training of the network, we know the ground truth 9D pose and use it to supervise the network to learn how to transform the observed point cloud into the canonical space.
The canonical coordinate space is pre-defined as in the paper NOCS.
SO during inference, we don't need the P_ori in canonical space

[vtasStu]

Thank you very much for your reply. I think there may be a sentence to try to explain my confusion:

Understanding the Limitations of CNN-based Absolute Camera Pose Regression (CVPR 2019)

We have also shown that APR(e2e method) is more closely related to image retrieval approaches than to methods that accurately estimate camera poses via 3D geometry.