2D joints from 3D prediction

[kallivad]

Hi @mkocabas, thank you a lot for your repo!
I wounder, if there any way of how to obtain 2D joints not from STAF tracker, but from 3D joints predicted by your net? Am I use pred_cam from https://github.com/mkocabas/VIBE/blob/master/doc/demo.md and how I can use it?

[mkocabas]

Hi @kallivad,

You can check this projection function:

VIBE/lib/models/spin.py

Line 442 in 3e04e0f

def perspective_projection(points, rotation, translation,

For a sample usage, check this:

VIBE/lib/smplify/losses.py

Line 78 in 3e04e0f

projected_joints = perspective_projection(model_joints, rotation, camera_t,

[mkocabas]

I am closing it now, feel free to ask if you need help about it.

[cbsudux]

Hey! I'm trying to get 2D pose from VIBE Output. How do I use the perspective projection function to do this? What inputs should I pass?

[RainkLH]

Same question.
For function "perspective_projection(points, rotation, translation,focal_length, camera_center)"
How to calculate Camera rotation、Camera translation、Focal length、Camera center

[tegusi]

For classical intrinsic and extrinsic matrix, the following solution works well.

cam = dicts['orig_cam']
cam_s = cam[0:1]
cam_pos = cam[2:]
flength = w / 2.
tz = flength / (0.5 * w * cam_s)
trans = -np.hstack([cam_pos, tz])
camera_data['color_focal_length'].append(np.array([w / 2, w / 2]))
camera_data['color_center'].append(np.array([[w / 2, h / 2]]))
camera_data['c2w'].append(np.eye(4))
camera_data['c2w'][:3,3] = trans

[lvZic]

For classical intrinsic and extrinsic matrix, the following solution works well.

cam = dicts['orig_cam']
cam_s = cam[0:1]
cam_pos = cam[2:]
flength = w / 2.
tz = flength / (0.5 * w * cam_s)
trans = -np.hstack([cam_pos, tz])
camera_data['color_focal_length'].append(np.array([w / 2, w / 2]))
camera_data['color_center'].append(np.array([[w / 2, h / 2]]))
camera_data['c2w'].append(np.eye(4))
camera_data['c2w'][:3,3] = trans

I found the cam params converge worse, and i use weak perspective in my code, in which kpy_2d = scale(kyp3d[, :2] + txy ). I think the key reason is the focal length of the dataset is different with each image, and it range from 400 mm to 800 mm. So maybe the network cannot regress the scale well?
As "It is common to assume a fixed focal length to perform perspective projection. " . I wonder if the performance would be improved if i use perspective projection instead of weak perspective?