MOT 17 evaluation (private detector)
Mike edited this page Nov 8, 2022
·
2 revisions
-
Place you people model under
Yolov5_StrongSORT_Pytorch/weights/<my_people_weights>. Notice that people should be class 0! -
Run the evaluation script
python val.py --yolo-weights <my_people_weights> --benchmark MOT17 --tracking-method strongsort
ocsort
bytetrackHOTA: exp350-pedestrian HOTA DetA AssA DetRe DetPr AssRe AssPr LocA RHOTA HOTA(0) LocA(0) HOTALocA(0)
MOT17-04-sso 60.903 58.993 63.403 64.114 76.01 68.894 78.917 80.979 63.71 81.253 75.154 61.065
MOT17-05-sso 40.221 39.015 41.585 42.038 73.858 52.674 64.36 81.464 41.802 52.011 76.225 39.645
MOT17-09-sso 56.987 60.303 53.896 65.706 80.47 59.27 79.875 85.73 59.499 70.723 81.904 57.925
MOT17-10-sso 50.526 50.106 51.144 53.923 74.797 56.125 76.017 80.42 52.508 67.457 75.495 50.926
MOT17-11-sso 62.713 58.618 67.31 70.413 72.726 74.494 81.952 86.644 68.834 75.206 83.095 62.492
MOT17-13-sso 46.574 42.694 51.2 46.504 72.051 57.049 74.184 80.166 48.743 61.587 74.636 45.966
COMBINED 56.783 54.276 59.96 59.626 75.202 66.076 78.529 81.811 59.71 74.077 76.461 56.64
CLEAR: exp350-pedestrian MOTA MOTP MODA CLR_Re CLR_Pr MTR PTR MLR sMOTA CLR_TP CLR_FN CLR_FP IDSW MT PT ML Frag
MOT17-04-sso 69.712 78.485 69.84 77.095 91.4 50.602 33.735 15.663 53.125 36664 10893 3450 61 42 28 13 544
MOT17-05-sso 45.251 78.771 46.219 51.569 90.602 20.301 59.398 20.301 34.303 3567 3350 370 67 27 79 27 249
MOT17-09-sso 67.887 84.473 68.695 75.174 92.065 46.154 53.846 0 56.215 4003 1322 345 43 12 14 0 141
MOT17-10-sso 62.287 76.987 62.933 67.513 93.647 29.825 59.649 10.526 46.75 8668 4171 588 83 17 34 6 659
MOT17-11-sso 61.562 85.284 61.912 79.366 81.972 45.333 41.333 13.333 49.882 7489 1947 1647 33 34 31 10 202
MOT17-13-sso 52.13 76.694 52.826 58.684 90.924 30.909 40 29.091 38.453 6832 4810 682 81 34 44 32 313
COMBINED 63.781 79.239 64.174 71.731 90.469 34.298 47.521 18.182 48.889 67223 26493 7082 368 166 230 88 2108
Identity: exp350-pedestrian IDF1 IDR IDP IDTP IDFN IDFP
MOT17-04-sso 77.33 71.279 84.504 33898 13659 6216
MOT17-05-sso 54.468 42.735 75.083 2956 3961 981
MOT17-09-sso 70.361 63.906 78.266 3403 1922 945
MOT17-10-sso 69.038 59.405 82.401 7627 5212 1629
MOT17-11-sso 74.359 73.177 75.58 6905 2531 2231
MOT17-13-sso 62.999 51.83 80.303 6034 5608 1480
COMBINED 72.399 64.901 81.856 60823 32893 13482
Count: exp350-pedestrian Dets GT_Dets IDs GT_IDs
MOT17-04-sso 40114 47557 131 83
MOT17-05-sso 3937 6917 109 133
MOT17-09-sso 4348 5325 53 26
MOT17-10-sso 9256 12839 97 57
MOT17-11-sso 9136 9436 154 75
MOT17-13-sso 7514 11642 132 110
COMBINED 74305 93716 676 484
HOTA: exp351-pedestrian HOTA DetA AssA DetRe DetPr AssRe AssPr LocA RHOTA HOTA(0) LocA(0) HOTALocA(0)
MOT17-04-oc 60.602 58.241 63.567 63.006 76.603 67.462 81.317 81.218 63.24 80.674 75.564 60.961
MOT17-05-oc 36.169 35.463 36.976 37.239 77.514 45.597 62.654 82.398 37.099 45.704 78.076 35.684
MOT17-09-oc 52.781 60.202 46.307 64.74 81.828 59.729 67.641 85.813 54.745 65.399 82.173 53.74
MOT17-10-oc 38.462 45.966 32.442 48.583 77.16 35.109 70.218 81.267 39.653 49.58 76.841 38.097
MOT17-11-oc 59.541 60.22 59.106 69.329 76.693 66.842 79.257 87.066 63.995 70.497 83.923 59.163
MOT17-13-oc 43.834 40.402 47.912 43.131 74.533 53.102 71.545 80.64 45.408 57.116 75.438 43.087
COMBINED 54.249 53.006 56.022 57.395 76.836 61.347 77.765 82.213 56.625 70.25 77.148 54.197
CLEAR: exp351-pedestrian MOTA MOTP MODA CLR_Re CLR_Pr MTR PTR MLR sMOTA CLR_TP CLR_FN CLR_FP IDSW MT PT ML Frag
MOT17-04-oc 69.422 78.946 69.554 75.903 92.282 48.193 36.145 15.663 53.441 36097 11460 3019 63 40 30 13 364
MOT17-05-oc 41.94 79.931 42.981 45.511 94.734 15.038 60.15 24.812 32.807 3148 3769 175 72 20 80 33 141
MOT17-09-oc 68.864 84.654 69.202 74.16 93.734 50 50 0 57.484 3949 1376 264 18 13 13 0 61
MOT17-10-oc 56.671 78.149 57.855 60.41 95.943 22.807 54.386 22.807 43.471 7756 5083 328 152 13 31 13 367
MOT17-11-oc 65.568 85.72 65.981 78.19 86.495 44 40 16 54.402 7378 2058 1152 39 33 30 12 102
MOT17-13-oc 50.318 77.435 51.065 54.467 94.122 23.636 46.364 30 38.028 6341 5301 396 87 26 51 33 235
COMBINED 62.854 79.871 63.314 69.005 92.38 29.959 48.554 21.488 48.964 64669 29047 5334 431 145 235 104 1270
Identity: exp351-pedestrian IDF1 IDR IDP IDTP IDFN IDFP
MOT17-04-oc 76.573 69.777 84.835 33184 14373 5932
MOT17-05-oc 48.047 35.565 74.029 2460 4457 863
MOT17-09-oc 64.605 57.859 73.131 3081 2244 1132
MOT17-10-oc 47.641 38.819 61.653 4984 7855 3100
MOT17-11-oc 69.387 66.056 73.072 6233 3203 2297
MOT17-13-oc 59.318 46.822 80.911 5451 6191 1286
COMBINED 67.668 59.107 79.129 55393 38323 14610
Count: exp351-pedestrian Dets GT_Dets IDs GT_IDs
MOT17-04-oc 39116 47557 137 83
MOT17-05-oc 3323 6917 108 133
MOT17-09-oc 4213 5325 37 26
MOT17-10-oc 8084 12839 125 57
MOT17-11-oc 8530 9436 158 75
MOT17-13-oc 6737 11642 117 110
COMBINED 70003 93716 682 484
HOTA: exp35-pedestrian HOTA DetA AssA DetRe DetPr AssRe AssPr LocA RHOTA HOTA(0) LocA(0) HOTALocA(0)
MOT17-04-FRCNN 59.049 57.471 61.146 61.714 76.964 65.139 80.225 81.072 61.38 78.235 75.759 59.27
MOT17-05-FRCNN 37.968 36.615 39.489 38.541 76.518 46.754 67.318 81.584 38.999 48.879 76.841 37.559
MOT17-09-FRCNN 53.414 60.071 47.548 64.353 80.555 57.084 70.268 84.041 55.303 68.569 79.796 54.715
MOT17-10-FRCNN 40.031 47.602 33.89 50.503 76.211 37.814 68.633 80.618 41.341 52.281 75.863 39.662
MOT17-11-FRCNN 57.01 60.26 54.189 67.987 77.685 61.854 77.595 86.216 60.675 68.28 83.111 56.748
MOT17-13-FRCNN 43.156 38.306 48.912 40.398 75.394 53.691 74.257 80.355 44.413 56.533 75.336 42.59
COMBINED 53.241 52.663 54.285 56.601 77.014 59.412 77.263 81.787 55.36 69.167 76.859 53.161
CLEAR: exp35-pedestrian MOTA MOTP MODA CLR_Re CLR_Pr MTR PTR MLR sMOTA CLR_TP CLR_FN CLR_FP IDSW MT PT ML Frag
MOT17-04-FRCNN 69.287 78.586 69.399 74.792 93.274 48.193 34.94 16.867 53.272 35569 11988 2565 53 40 29 14 311
MOT17-05-FRCNN 44.123 78.808 45.309 47.839 94.977 20.301 54.887 24.812 33.985 3309 3608 175 82 27 73 33 134
MOT17-09-FRCNN 70.178 82.334 70.873 75.38 94.358 53.846 42.308 3.8462 56.862 4014 1311 240 37 14 11 1 73
MOT17-10-FRCNN 59.202 77.415 60.332 63.299 95.522 26.316 52.632 21.053 44.906 8127 4712 381 145 15 30 12 435
MOT17-11-FRCNN 67.645 84.559 68.037 77.777 88.871 44 36 20 55.636 7339 2097 919 37 33 27 15 66
MOT17-13-FRCNN 48.033 76.93 48.48 51.031 95.239 20.909 40.909 38.182 36.26 5941 5701 297 52 23 45 42 133
COMBINED 63.293 79.212 63.727 68.61 93.355 31.405 44.421 24.174 49.031 64299 29417 4577 406 152 215 117 1152
Identity: exp35-pedestrian IDF1 IDR IDP IDTP IDFN IDFP
MOT17-04-bt 75.228 67.776 84.523 32232 15325 5902
MOT17-05-bt 51.38 38.629 76.693 2672 4245 812
MOT17-09-bt 67.126 60.376 75.576 3215 2110 1039
MOT17-10-bt 49.421 41.086 62 5275 7564 3233
MOT17-11-bt 66.452 62.304 71.192 5879 3557 2379
MOT17-13-bt 58.747 45.113 84.194 5252 6390 986
COMBINED 67.07 58.181 79.164 54525 39191 14351
Count: exp35-pedestrian Dets GT_Dets IDs GT_IDs
MOT17-04-bt 38134 47557 111 83
MOT17-05-bt 3484 6917 117 133
MOT17-09-bt 4254 5325 36 26
MOT17-10-bt 8508 12839 130 57
MOT17-11-bt 8258 9436 127 75
MOT17-13-bt 6238 11642 102 110
COMBINED 68876 93716 623 484