Segmentation model vs object detetion model on MOT metrics
Mike edited this page Dec 22, 2022
·
6 revisions
HOTA: exp333-pedestrian HOTA DetA AssA DetRe DetPr AssRe AssPr LocA OWTA HOTA(0) LocA(0) HOTALocA(0)
MOT17-04-FRCNN 48.026 41.227 56.245 44.678 76.993 60.018 83.968 84.253 50.09 62.296 76.817 47.854
MOT17-05-FRCNN 40.099 39.414 41.327 45.198 63.145 50.738 62.782 76.957 43.138 59.225 66.514 39.393
MOT17-09-FRCNN 49.958 54.709 45.795 61.094 75.552 51.28 75.088 83.485 52.886 64.615 78.253 50.563
MOT17-10-FRCNN 46.113 46.385 46.021 50.54 71.788 51.213 73.212 79.139 48.219 63.915 72.6 46.402
MOT17-11-FRCNN 54.829 49.251 61.381 60.395 67.211 67.402 82.495 84.224 60.857 70.917 76.47 54.23
MOT17-13-FRCNN 41.167 34.433 49.517 36.534 74.319 54.577 74.772 80.885 42.492 53.459 75.688 40.462
COMBINED 47.479 42.552 53.446 47.023 73.331 58.48 79.954 82.554 50.056 62.434 75.29 47.007
CLEAR: exp333-pedestrian MOTA MOTP MODA CLR_Re CLR_Pr MTR PTR MLR sMOTA CLR_TP CLR_FN CLR_FP IDSW MT PT ML Frag
MOT17-04-FRCNN 40.213 83.836 40.371 49.2 84.785 20.482 45.783 33.735 32.26 23398 24159 4199 75 17 38 28 432
MOT17-05-FRCNN 32.861 74.732 34.22 52.899 73.904 17.293 63.158 19.549 19.495 3659 3258 1292 94 23 84 26 226
MOT17-09-FRCNN 61.446 81.816 62.347 71.606 88.551 46.154 50 3.8462 48.425 3813 1512 493 48 12 13 1 65
MOT17-10-FRCNN 53.766 75.986 54.42 62.411 88.649 31.579 50.877 17.544 38.779 8013 4826 1026 84 18 29 10 334
MOT17-11-FRCNN 42.9 83.879 43.122 66.49 73.995 33.333 26.667 40 32.181 6274 3162 2205 21 25 20 30 63
MOT17-13-FRCNN 40.947 77.701 41.651 45.405 92.364 23.636 36.364 40 30.822 5286 6356 437 82 26 40 44 214
COMBINED 43.095 81.139 43.526 53.825 83.939 25 46.281 28.719 32.943 50443 43273 9652 404 121 224 139 1334
Identity: exp333-pedestrian IDF1 IDR IDP IDTP IDFN IDFP
MOT17-04-FRCNN 56.412 44.574 76.813 21198 26359 6399
MOT17-05-FRCNN 51.55 44.224 61.785 3059 3858 1892
MOT17-09-FRCNN 60.409 54.629 67.557 2909 2416 1397
MOT17-10-FRCNN 60.819 51.819 73.603 6653 6186 2386
MOT17-11-FRCNN 62.238 59.082 65.751 5575 3861 2904
MOT17-13-FRCNN 54.466 40.62 82.631 4729 6913 994
COMBINED 57.373 47.082 73.422 44123 49593 15972
Count: exp333-pedestrian Dets GT_Dets IDs GT_IDs
MOT17-04-FRCNN 27597 47557 148 83
MOT17-05-FRCNN 4951 6917 164 133
MOT17-09-FRCNN 4306 5325 77 26
MOT17-10-FRCNN 9039 12839 122 57
MOT17-11-FRCNN 8479 9436 151 75
MOT17-13-FRCNN 5723 11642 127 110
COMBINED 60095 93716 789 484
HOTA: exp342-pedestrian HOTA DetA AssA DetRe DetPr AssRe AssPr LocA OWTA HOTA(0) LocA(0) HOTALocA(0)
MOT17-04-FRCNN 46.268 40.507 53.084 43.65 77.509 57.073 82.028 84.142 48.113 60.324 76.513 46.156
MOT17-05-FRCNN 39.402 39.461 39.945 45.312 63.063 48.639 63.758 76.969 42.436 57.683 66.837 38.553
MOT17-09-FRCNN 51.775 55.197 48.674 61.194 76.153 53.973 78.013 83.43 54.574 67.121 78.622 52.772
MOT17-10-FRCNN 46.507 45.036 48.21 48.834 72.091 51.766 75.005 79.146 48.512 64.044 72.683 46.549
MOT17-11-FRCNN 55.933 49.806 63.148 59.653 69.347 68.28 83.685 84.404 61.35 71.674 76.957 55.158
MOT17-13-FRCNN 40.322 32.507 50.289 34.156 76.089 55.286 75.748 81.533 41.408 51.569 76.745 39.577
COMBINED 46.762 41.864 52.629 45.911 74.093 57.331 79.778 82.589 49.1 61.404 75.366 46.277
CLEAR: exp342-pedestrian MOTA MOTP MODA CLR_Re CLR_Pr MTR PTR MLR sMOTA CLR_TP CLR_FN CLR_FP IDSW MT PT ML Frag
MOT17-04-FRCNN 39.693 83.844 39.847 48.081 85.378 20.482 40.964 38.554 31.926 22866 24691 3916 73 17 34 32 458
MOT17-05-FRCNN 32.745 74.701 34.263 53.058 73.843 19.549 63.91 16.541 19.322 3670 3247 1300 105 26 85 22 216
MOT17-09-FRCNN 62.085 81.533 62.629 71.493 88.969 46.154 50 3.8462 48.882 3807 1518 472 29 12 13 1 63
MOT17-10-FRCNN 52.582 75.999 53.127 60.433 89.215 28.07 49.123 22.807 38.078 7759 5080 938 70 16 28 13 314
MOT17-11-FRCNN 45.369 84.02 45.581 65.801 76.494 32 29.333 38.667 34.854 6209 3227 1908 20 24 22 29 51
MOT17-13-FRCNN 38.808 78.53 39.272 42.08 93.743 20.909 34.545 44.545 29.773 4899 6743 327 54 23 38 49 176
COMBINED 42.68 81.24 43.055 52.51 84.741 24.38 45.455 30.165 32.829 49210 44506 8861 351 118 220 146 1278
Identity: exp342-pedestrian IDF1 IDR IDP IDTP IDFN IDFP
MOT17-04-FRCNN 53.813 42.059 74.684 20002 27555 6780
MOT17-05-FRCNN 51.232 44.022 61.268 3045 3872 1925
MOT17-09-FRCNN 64.89 58.516 72.821 3116 2209 1163
MOT17-10-FRCNN 61.441 51.53 76.072 6616 6223 2081
MOT17-11-FRCNN 64.866 60.333 70.137 5693 3743 2424
MOT17-13-FRCNN 52.359 37.932 84.501 4416 7226 810
COMBINED 56.511 45.764 73.854 42888 50828 15183
Count: exp342-pedestrian Dets GT_Dets IDs GT_IDs
MOT17-04-FRCNN 26782 47557 145 83
MOT17-05-FRCNN 4970 6917 155 133
MOT17-09-FRCNN 4279 5325 59 26
MOT17-10-FRCNN 8697 12839 104 57
MOT17-11-FRCNN 8117 9436 146 75
MOT17-13-FRCNN 5226 11642 113 110
COMBINED 58071 93716 722 484
HOTA -0.717
MOTA -0.415
IDF1 -0.862
Results are very similar on all main MOT metrics, a bit worse for the seg model as can be sen above. Seg models are in average 5 times slower