The keypoint confidences are more than 1

[dmvictor]

I ran the demo script to run mmdet for human detection, and mmpose for pose estimation. I have tried two models and printed the poseResults. The results made me confused.

The first model is 3xrsn_50, the poseResults showed as below:

 [{'bbox': array([421.7143, 224.3, 620.24994, 933.0151, 0.99870956], dtype=float32), 
'keypoints': array([[552.6708   , 298.45966  ,   1.2614805],
       [552.6708   , 284.59802  ,   1.2813547],
       [538.8091   , 284.59802  ,   1.2185569],
       [552.6708   , 291.52887  ,   1.0129664],
       [497.22427  , 298.45966  ,   1.2192444],
       [559.60156  , 374.69855  ,   1.1950808],
       [469.501    , 395.49103  ,   1.1472647],
       [573.46313  , 471.72992  ,   1.1317949],
       [462.57022  , 506.38397  ,   1.1898675],
       [594.2556   , 554.8997   ,   1.1886529],
       [497.22427  , 603.41534  ,   1.1690912],
       [559.60156  , 589.5538   ,   1.0519567],
       [504.15506  , 603.41534  ,   1.0438308],
       [545.74     , 735.10077  ,   1.166288 ],
       [497.22427  , 748.96246  ,   1.1420388],
       [545.74     , 859.8554   ,   1.195259 ],
       [497.22427  , 880.64777  ,   1.1333697]], dtype=float32)}]

The second model is 4xmspn_50, and the poseResults showed as below:

[{'bbox': array([421.7143, 224.3, 620.24994, 933.0151, 0.99870956], dtype=float32), 
'keypoints': array([[552.6708   , 298.45966  ,   1.2303721],
       [552.6708   , 284.59802  ,   1.264624 ],
       [538.8091   , 284.59802  ,   1.261019 ],
       [552.6708   , 284.59802  ,   1.0062089],
       [497.22427  , 298.45966  ,   1.2029631],
       [559.60156  , 374.69855  ,   1.2056566],
       [469.501    , 388.56024  ,   1.1578579],
       [573.46313  , 471.72992  ,   1.1706965],
       [462.57022  , 506.38397  ,   1.1652234],
       [594.2556   , 554.8997   ,   1.226975 ],
       [497.22427  , 603.41534  ,   1.1875193],
       [559.60156  , 596.48456  ,   1.1021883],
       [497.22427  , 603.41534  ,   1.1097052],
       [545.74     , 742.03156  ,   1.1680834],
       [497.22427  , 748.96246  ,   1.1607904],
       [545.74     , 859.8554   ,   1.1985698],
       [497.22427  , 880.64777  ,   1.1731863]], dtype=float32)}]     

The keypoint confidences of these two models are more than 1.
Is there any problem with the result?
If there is no problem, how could we know the range of the confidence?
Thank you!

[jin-s13]

Yes, it is possible. We did not explicitly restrict the output to be [0, 1].

[dmvictor]

Thanks for your response! Is there any way to know the boundary confidence value of a specified model?

[jin-s13]

Most our models are trained using "MSRA-type" encoding and decoding methods for heatmaps. The values of heatmaps are between [0, 1] during training.
But MSPN and RSN are different. They adopt "Megvii-type" encoding and decoding methods for heatmaps. The values of heatmaps are between [0, 255] during training. And during decoding, the scores will be divided by 255.

I guess that is why the output of RSN will exceed 1.

[julianzille]

Hi @jin-s13 - I'm also having trouble understanding confidence scores, especially when they exceed 1 (or 255 before decoding). Could you perhaps point me to any resources that may help me interpret the confidence scores? If they are not on a fixed scale, I don't see how they are meaningful? With thanks!

[dmvictor]

In your assumption, the confidences of RSN exceed 1 imply that the intermediate results during decoding exceed 255. Is it possible to dump the intermediate results during decoding?

[jin-s13]

Yes, we can obtain the heatmaps.

mmpose/demo/top_down_img_demo.py

Line 98 in 2a0a2d2

pose_results, returned_outputs = inference_top_down_pose_model(

[smidm]

Can you refer to a paper where is the "Megvii-type" encoding and decoding method described?

[liqikai9]

Hi, you can refer to the paper of CPN and MSPN.