Problems about the results of MOT17

[SpyderXu]

It is great work to open source code in MOT field. I really appreciate it. However I find that the tracking results with DPM is same with that with SDP in MOT17 ranking list about your tracker. Are there something wrong with the system?

[shijieS]

Thank you for your problem. I'll update the result after finishing the README.md part.

[yihongXU]

Hi, Thank you so much for your open source code which is really rare in MOT community. However I ran into the same issue as @SpyderXu , I used the test_mot17.py to produce results for MOT17 trainset and calculate the MOTA score (tools from https://github.com/cheind/py-motmetrics), with a standard threshold IOU 0.5 (same as the MOTchanllenge). I used the weight that you shared : sst300_0712_83000.pth and I have got the following results:
MOT17-11-DPM
MOTP MOTA FP FN ID_SW num_objects
final 0.219081 43.07% 794 4492 86 9436
MOT17-02-DPM
MOTP MOTA FP FN ID_SW num_objects
final 0.251454 14.32% 697 15052 171 18581
MOT17-11-FRCNN
MOTP MOTA FP FN ID_SW num_objects
final 0.118502 56.44% 303 3740 67 9436
MOT17-02-SDP
MOTP MOTA FP FN ID_SW num_objects
final 0.200413 32.22% 2656 9621 318 18581
MOT17-09-SDP
MOTP MOTA FP FN ID_SW num_objects
final 0.162181 60.79% 145 1868 75 5325
MOT17-10-SDP
MOTP MOTA FP FN ID_SW num_objects
final 0.227044 59.12% 1229 3724 296 12839
MOT17-13-DPM
MOTP MOTA FP FN ID_SW num_objects
final 0.320541 7.62% 602 10069 84 11642
MOT17-05-DPM
MOTP MOTA FP FN ID_SW num_objects
final 0.26054 25.63% 422 4608 114 6917
MOT17-04-SDP
MOTP MOTA FP FN ID_SW num_objects
final 0.151199 73.87% 919 11362 145 47557
MOT17-05-SDP
MOTP MOTA FP FN ID_SW num_objects
final 0.190729 53.78% 413 2567 217 6917
MOT17-10-FRCNN
MOTP MOTA FP FN ID_SW num_objects
final 0.197432 41.55% 2012 5169 323 12839
MOT17-04-FRCNN
MOTP MOTA FP FN ID_SW num_objects
final 0.098745 51.56% 1891 21068 79 47557
MOT17-10-DPM
MOTP MOTA FP FN ID_SW num_objects
final 0.263786 27.18% 815 8437 97 12839
MOT17-13-FRCNN
MOTP MOTA FP FN ID_SW num_objects
final 0.270735 29.38% 2295 5567 359 11642
MOT17-09-DPM
MOTP MOTA FP FN ID_SW num_objects
final 0.267049 37.43% 865 2329 138 5325
MOT17-02-FRCNN
MOTP MOTA FP FN ID_SW num_objects
final 0.125864 26.27% 1593 12001 106 18581
MOT17-13-SDP
MOTP MOTA FP FN ID_SW num_objects
final 0.281644 31.45% 1837 5842 302 11642
MOT17-04-DPM
MOTP MOTA FP FN ID_SW num_objects
final 0.217009 32.21% 3586 28322 330 47557
MOT17-05-FRCNN
MOTP MOTA FP FN ID_SW num_objects
final 0.20983 47.71% 222 3294 101 6917
MOT17-11-SDP
MOTP MOTA FP FN ID_SW num_objects
final 0.165088 66.19% 561 2511 118 9436
MOT17-09-FRCNN
MOTP MOTA FP FN ID_SW num_objects
final 0.124762 55.44% 30 2310 33 5325
avg mota: 43.187 %
total fn: 163953.0
total fp: 23887.0
total idsw: 3559.0
total_num_objects: 336891.0

It there a way to improve the results, please.

Thank you a again for your contribution.

[yihongXU]

Did you try to submit your results for MOT challenge. If so, could you indicate please the name of your method from the ranking board of MOT17 ?

[miooooooooooo]

@yihongXU hi,I find the paper's results for MOT17 is in the ranking board where the detector is private.And can you repetition the paper's results?Because I don't know which detection the author used,because Faster R-CNN and DPM which mentioned in the paper can't reach the paper's scores