How can I get the 21 hand key points ?

[hungsing92]

No description provided.

[quyanqiu]

If you means the 2d hand keypoints, just use openpose (https://github.com/CMU-Perceptual-Computing-Lab/openpose) to detect it.
If you means the 3d hand keypoints, you need use MANO hand model, Find the Joint regressor matrix, the original mano hand model only regress 16 3d hand joints,Assume your verts is V=778X3, regressor=R=16X778,you can easily obtain 3d hand joints by simple matrix multiply: J=RXV=16X3,the extra five finger tips are sampled from mano mesh surface.
There is a differentiable mano layer here（https://github.com/hassony2/manopth）,you may check this page for more details

[hungsing92]

Thanks for your quick reply. Would you plan to release the mano parameters?

[quyanqiu]

@hungsing92,Unfortunately,This is not my project lol, But you could register your mano model to these vertex data by ICP algorithm

[dkulon]

@hungsing92

To get 3D keypoints, you can use the joint regressor as @quyanqiu suggested from MANO (https://mano.is.tue.mpg.de) that regresses 16 joints from the mesh surface. To get fingertips, you can select vertices with the following indices: FINGERTIP_IDXS_MANO = [744, 320, 443, 554, 671].

Meshes are aligned with the images so to obtain 2D keypoints, you simply perform orthographic projection of the 3D joints (i.e. X, Y coordinates of the 3D joints correspond to 2D image landmarks).

We are not planning to release any further data.

[SuhelNaryal]

@hungsing92
Hi, If you have downloaded the images and figured how to get 21 3d keypoints then can you share both of them with me, please?