Metric list , adding new metric module

[greg-is-kub]

Hi !

I'm currently trying to do a benchmark of the coco dataset with the folowing metrics : PDJ , OKS and PCK , but I haven't found a way to see if they were available in mmpose nor a if i could add a metric in mmpose as a module.

Thanks in advance for your help and thank you for developping this really useful toolbox.

greetings

[jin-s13]

Please check

mmpose/mmpose/datasets/datasets/top_down/topdown_coco_dataset.py

Line 332 in c8ff23f

allowed_metrics = ['mAP']

Currently, we only support AP for COCO. Different datasets have different allowed_metrics. Maybe we should make the error-info more clear.

[greg-is-kub]

Thank you for your help ! I'll try to carry on from here for the implementation 👍

I only have one question left : the value used for threshold to differenciate positive from negative for the mAP metric for coco is it the OKS metric ?

[jin-s13]

Yes, we used OKS metric.

[greg-is-kub]

Thanks a lot !

[julianzille]

Hi @greg-is-kub
I'm also trying to evaluate my custom dataset using PCK. Were you successful in adding this as an acceptable metric?