Wrong results

[phj128]

Hi, when I ran your test on validation, I got such results:
Evaluate class: Cars
sMOTSA MOTSA MOTSP MOTSAL MODSA MODSP Recall Prec F1 FAR MT PT ML TP FP FN IDS Frag GT Obj GT Trk TR Obj TR Trk Ig TR Tck
all 83.87 93.31 90.26 95.18 95.20 92.39 96.86 98.32 97.58 4.46 94.70 5.30 0.00 7777 133 252 152 219 8029 151 10097 109 2187
0002 76.16 90.03 85.22 91.56 91.69 87.16 93.91 97.70 95.77 8.58100.00 0.00 0.00 848 20 55 15 30 903 15 1201 11 333
0006 89.56 97.58 91.89 97.77 97.77 92.75 98.88 98.88 98.88 2.22100.00 0.00 0.00 531 6 6 1 3 537 11 756 7 219
0007 87.86 94.51 93.31 97.97 98.05 94.65 99.38 98.68 99.03 3.75 98.11 1.89 0.00 2244 30 14 80 82 2258 53 3159 29 885
0008 88.39 96.74 91.48 97.51 97.60 91.32 97.98 99.61 98.79 1.03 90.48 9.52 0.00 1021 4 21 9 16 1042 21 1216 15 191
0010 88.54 96.35 92.02 97.06 97.18 92.72 97.84 99.33 98.58 1.36 92.31 7.69 0.00 589 4 13 5 8 602 13 659 12 66
0013 -12.12 -5.56 92.84 -2.78 -2.78 99.30 91.67 49.25 64.08 10.00100.00 0.00 0.00 33 34 3 1 1 36 2 124 6 57
0014 76.24 87.36 88.37 94.22 94.55 88.66 95.64 98.87 97.23 4.72 85.71 14.29 0.00 439 5 20 33 36 459 14 552 9 108
0016 70.40 88.97 79.62 89.09 89.09 79.58 91.13 97.81 94.35 8.13100.00 0.00 0.00 760 17 74 1 37 834 4 783 10 6
0018 88.00 95.14 92.61 95.59 95.66 93.44 96.61 99.02 97.80 3.83 88.89 11.11 0.00 1312 13 46 7 6 1358 18 1647 10 322

This is not consistent with the claimed one. sMOTSA and MOTSA are similar, but IDS is 152.
Is there anything wrong with the test process or the model is not right?
Thank you for your awesome work.

[anirudh-chakravarthy]

Hi,

I got the same result using the pre-trained weights. Were you able to solve this?

@detectRecog could you please explain why this performance difference is observed?

[phj128]

No

[EchoAmor]

@detectRecog I got the same problem ,mine is as follows, why there appeared -12 ?? can u please explain this situation?

[Ilyabasharov]

@EchoAmor @phj128, Did u carefully examine the 13th track, which consists of much more pedestrians than cars? so, your model often makes mistakes on this track. Look at precision metric, lots of false positives.

[EchoAmor]

@Ilyabasharov oh,thanks a lot! I didn't check the images before

[Ilyabasharov]

@EchoAmor How did you achieve such results ? Here #15 you wrote that u have lower results.

[EchoAmor]

@Ilyabasharov this result is run with the author's pretrained weight, and #15 is trained from the initial by myself.

[qa276390]

Hi @EchoAmor! did you get the result of IDS=21 they report on paper?