How to mapping from xy in joint_2d to x'y' in joint_3d_mono?

[Fangyh09]

Hi, is there any relationship between x,y in pts and x,y in pts_3d_mono?
The model output x,y is in pts, but the ground truth is in pts_3d_mono. So I want to figure out whether there is a mapping. Thanks!

pytorch-pose-hg-3d/src/datasets/h36m.py

Lines 40 to 43 in 84ad44e

	def GetPartInfo(self, index):
	pts = self.annot['joint_2d'][index].copy()
	pts_3d_mono = self.annot['joint_3d_mono'][index].copy()
	pts_3d = self.annot['joint_3d_mono'][index].copy()

[xingyizhou]

Hi,
This is a good question. The detailed transformation code from image xy to world xy is here https://github.com/xingyizhou/pytorch-pose-hg-3d/blob/master/src/utils/eval.py#L70, or a more detailed explanation at xingyizhou/pose-hg-3d#3 . Also, we provide the detailed projection formulation at Section 3.2 of https://arxiv.org/pdf/1803.09331.pdf.

[Fangyh09]

Thanks @xingyizhou
I want to know that during evaluation whether we could use the extrinsic parameters or not?

[xingyizhou]

Yes, you can use camera parameters to perform the full perspective projection. However, the depth of the root should be known to resolve the scale-depth ambiguity.

[Fangyh09]

@xingyizhou
It uses gt meta[i, root] in L84, so the error is relative error instead of absolute error?
I think the difference between protocol I and protocol II is the procruste instead of using gt meta[i, root]. Is it right? Thanks~

pytorch-pose-hg-3d/src/utils/eval.py

Line 84 in 84ad44e

p[i, j] = (p[i, j] - pRoot) / lenPred * h36mSumLen + meta[i, root]

[xingyizhou]

Hi,
Sorry for the very delayed reply. I think all evaluations on H36M are based on relative depth. You can check the official H36M ECCV Challenge data and its evaluation script.