Difference of joints

[ziqipang]

Hi @akanazawa It's really great job and helps me a lot.
However, when I try to get the joints of human and use them for other computer vision tasks, I'm confused by the actual meaning of joints, joints3d and joints_ori.

So I'm curious to know, if I directly use joints and joints3d, will they match the joints on the original 2D image?

Thanks a lot!

[akanazawa]

Hi @xingyueyaohui,

Thank you!
Sure, the naming is not so great here either, here's what they are:
joints: this is the K x 2, 2D joints in the normalized image coordinate space. Normalized image coordinate space treats image to have range [-1, 1] x [-1, 1] no matter what the size of image was.
joints3d: K x 3, 3D joints in the SMPL coordinate space.

Therefore neither joints nor joints3d will match the original 2D image.

See how I convert from normalized image coordinate space to image coordinate space here.
After this transformation, the joints should match the original 2D image (up to prediction error).

In order to convert the joints3d to a normalized image coordinate space (joints), you have to project the 3D points using the predicted camera. This is done in this projection function

Hope this clears things.

Best,

Angjoo

[ziqipang]

Thanks a lot, your answer is really helpful!

[A7ocin]

Thanks for the clarifications, @akanazawa , really helpful
It is not still clear to me what is the SMPL coordinate space unit of measurement. Is it meters perhaps?
Thank you!

[akanazawa]

Np! Yes I believe it is meters!

…

On Apr 1, 2019, at 8:23 AM, A7ocin ***@***.***> wrote: Thanks for the clarifications, @akanazawa , really helpful It is not still clear to me what is the SMPL coordinate space unit of measurement. Is it meters perhaps? Thank you! — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[A7ocin]

Ok, this means that the system is also able to retrieve an estimation of the person's height in meters?

[akanazawa]

Yes technically but note shape and absolute scale retrieval is a very challenging problem and the coco dataset that this is trained on has no ground truth 3D/height information :) so there's a lot more work to be done in this direction! Angjoo

…

On Apr 3, 2019, at 9:18 AM, A7ocin ***@***.***> wrote: Ok, this means that the system is also able to retrieve an estimation of the person's height in meters? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.