Is the metric or something wrong?

[aki34]

Thank you for sharing your code.

I pulled your code repository and set up the Human3.6M dataset according to your instruction.
After that, I'd like to test "Algebraic triangulation" so ran the following command:
python3 train.py --eval --eval_dataset train --config eval/human36m_alg.yaml --logdir ./logs
And then, I got the following message:

args: Namespace(config='eval/human36m_alg.yaml', eval=True, eval_dataset='train', local_rank=None, logdir='./logs', seed=42)
Number of available GPUs: 1
Loading pretrained weights from: ./data/pretrained/human36m/pose_resnet_4.5_pixels_human36m.pth
Reiniting final layer filters: module.final_layer.weight
Reiniting final layer biases: module.final_layer.bias
Successfully loaded pretrained weights for backbone
Successfully loaded pretrained weights for whole model
Loading data...
Experiment name: eval_human36m_alg_AlgebraicTriangulationNet@11.11.2019-21:14:55
/home/akihiko/MV3Dpose/learnable-triangulation-pytorch/mvn/datasets/human36m.py:220: RuntimeWarning: invalid value encountered in true_divide
action_scores[k] = v['total_loss'] / v['frame_count']
/home/akihiko/MV3Dpose/learnable-triangulation-pytorch/mvn/datasets/human36m.py:220: RuntimeWarning: invalid value encountered in double_scalars
action_scores[k] = v['total_loss'] / v['frame_count']
Done.

It seems to work but I got a huge error.
For example, the average "per pose error" is about 1200 and it means 1200 mm, doesn't it?

Thank you.

[karfly]

@aki34 sorry for long. Still working on the problem.

[aki34]

OK, I got it.

Thank you.

[TonyFaith]

@aki34 would you please share some Human3.6M datasets, that will be very appreciated!

[ericzw]

@aki34
v['frame_count'] could be zero. I obatin right results by adding the following codes.
for k, v in action_scores.items():
if v['frame_count'] > 0:
action_scores[k] = v['total_loss'] / v['frame_count']
else:
action_scores[k] = 0

[karfly]

Hi, @aki34.
We found out that the evaluation process wasn't totally correct. Please check out this issue for details.

[shrubb]

@aki34 I just pushed a large reproducibility update, please check if it brings you a lower error.