Questions about the Jacobian matrix

[lyj9494]

Congratulations on winning the honor of cvpr2022 best student paper and thanks for sharing your work!

However, the computation of the Jacobian matrix confused me a lot. In your code,

EPro-PnP/epropnp/camera.py

Line 121 in 1545131

d_x2d_d_x3dcam = torch.cat(

When the dof equals to 6,

d_x3dcam_d_rot = skew(x3d_rot * 2)
jac = torch.cat((d_x2d_d_x3dcam, d_x2d_d_x3dcam @ d_x3dcam_d_rot), dim=-1)

Could you please tell me how do you get the Jacobian matrix? It’s a little difficult for me to understand the d_x3dcam_d_rot, and I wish your reply could help me to figure out why the x3d_rot should be multiplied by 2, and why only the rotation is considered.

Additionally, I referred to some materials when trying to compute the Jacobian matrix, for example, in this article https://zhuanlan.zhihu.com/p/482540286, they compute the Jacobian matrix of the reprojection errors in this way:

I think if I follow this equation, the code should be

x3d_cam = x3d @ quaternion_to_rot_mat(pose[..., 3:]).transpose(-1, -2)+ pose[..., None, :3]
jac = torch.cat((d_x2d_d_x3dcam, d_x2d_d_x3dcam @ skew(x3d_cam)), dim=-1)

Which is different from your algorithm, and it puzzles me a lot. I am looking forward to your reply, and thank you again for your great job.

[Lakonik]

I did not follow the perturbation model mentioned in the article when writing the code. So perhaps our definition of orientation is different. The d_rot in this repo refers to a local delta in the 3D tangent space of the 4D quaternion hypersphere, and its conversion follows this transformation method:

EPro-PnP/epropnp/camera.py

Line 146 in 1545131

def get_quaternion_transfrom_mat(quaternions):

So our implementation is probably not standard practice, but still solves the problem.

[lyj9494]

Thank you for your patient reply ! Know I can figure out the difference between the two algorithms. Although I have a better understanding of d_x3dcam_d_rot, I am still confused that why it equals to skew(x3d_rot * 2), could you please tell me how to compute it?

[Lakonik]

Sorry but I did not write down the derivation of the Jacobian and I can't remember these details :(
But in fact it's not that difficult. You may try some symbolic math tools (like MATLAB) to get the results without doing it manually.

[lyj9494]

Thank you for your response !

[TruongKhang]

Hello @lyj9494 , I'm also confused about the Jacobian computation in the code. Did you figure out the answer to this?