Questions about training IKNet

[Gel-smile]

Thank you for great project，I have a few questions about training IKNet

1. When changing the original 16 rotations of MANO into 21 rotations, do W, T0, I0, M0, R0, and L0 share the rotation of W in the original MANO?
2. I found the joints_xyz calculated by using MANO ref_pose and the transformed 21 rotation parameters using the method in hand_mesh.py is not equal to the 'J_transformed' saved in the MANO pkl file , the order of joints has been adjusted according to kinematics.py. When using the MANO dataset to train IKNet, how did you get the ground truth 3D joint annotation in Lxyz? Is the calculation method of FK (Q) the same as the calculation method of joint_xyz in hand_mesh.py

[CalciferZh]

1. Yes that's right.
2. It's simply applying the rotations to each of the bones.

[Gel-smile]

Thank you for your reply. I have some questions about the second question. Do you calculate the joints_xyz by above method? if so, Do you use the ground truth of rotation and ref_pose to get the ground truth of joints_xyz in MANO datasets by above method?

[CalciferZh]

yes

[Gel-smile]

Using 'j_regressor' in hand_mesh_model.py and mesh vertex coordinates in the pkl file of MANO datasets can also calculate joints_xyz, but the result calculated by this method is inconsistent with the result obtained by the method mentioned above. Have you noticed this problem? Does this have no effect on the results?

[CalciferZh]

You may check the idea of linear blend skinning which would be helpful to understand where the difference comes from. The j_regressor and the vertices are all about skinning and is not really related to the IK part. In theory the output rotations can be applied to an arbitrary mesh model. We are only concerned about the joint angles in MANO.

[Gel-smile]

alright, thank you very much

[Gel-smile]

The model I trained does not converge, Could you share the training code of IKNet?

[CalciferZh]

We now have a different colleague working on v2 to release the training code. We are not sure when it will be ready.

[Gel-smile]

ok