Got very high robustness score when evaluating on VOT2018

[SampsonKwan]

Thanks for your great job first. However, when i evaluating the ATOM using the atom_default.pth checkpoint, I got very high robustness score. And when i check the visualized result of the bad cases, they shows that the tracker lost the target.

eval_result:

|------------------------------------------------------------
|Tracker Name| Accuracy | Robustness | Lost Number | EAO |
|------------------------------------------------------------
| atom_ | 0.602 | 2.852 | 609.0 | 0.037 |
|------------------------------------------------------------

details:

Done[1]: drone_flip[113] Lost: 18
Done[2]: fernando[292] Lost: 1
Done[3]: soccer2[129] Lost: 1
Done[4]: helicopter[708] Lost: 0
Done[5]: dinosaur[326] Lost: 2
Done[6]: iceskater1[661] Lost: 0
Done[7]: handball2[402] Lost: 13
Done[8]: book[175] Lost: 0
Done[9]: bolt2[293] Lost: 1
Done[10]: tiger[365] Lost: 0
Done[11]: shaking[365] Lost: 0
Done[12]: drone1[355] Lost: 53
Done[13]: ball1[105] Lost: 0
Done[14]: rabbit[158] Lost: 24
Done[15]: bolt1[350] Lost: 56
Done[16]: sheep[251] Lost: 0
Done[17]: glove[120] Lost: 6
Done[18]: ball2[41] Lost: 1
Done[19]: birds1[339] Lost: 38
Done[20]: godfather[366] Lost: 2
Done[21]: gymnastics3[118] Lost: 9
Done[22]: crossing[131] Lost: 0
Done[23]: conduction1[345] Lost: 12
Done[24]: leaves[63] Lost: 5
Done[25]: crabs1[160] Lost: 25
Done[26]: iceskater2[707] Lost: 4
Done[27]: blanket[225] Lost: 0
Done[28]: gymnastics2[240] Lost: 0
Done[29]: pedestrian1[140] Lost: 0
Done[30]: road[558] Lost: 0
Done[31]: motocross2[61] Lost: 0
Done[32]: basketball[725] Lost: 40
Done[33]: graduate[844] Lost: 0
Done[34]: frisbee[248] Lost: 0
Done[35]: motocross1[164] Lost: 1
Done[36]: bmx[76] Lost: 0
Done[37]: car1[742] Lost: 0
Done[38]: soccer1[392] Lost: 17
Done[39]: wiper[341] Lost: 0
Done[40]: fish1[366] Lost: 34
Done[41]: zebrafish1[400] Lost: 49
Done[42]: hand[267] Lost: 1
Done[43]: butterfly[151] Lost: 0
Done[44]: drone_across[147] Lost: 16
Done[45]: fish3[519] Lost: 0
Done[46]: ants3[579] Lost: 7
Done[47]: singer2[366] Lost: 5
Done[48]: soldier[138] Lost: 5
Done[49]: bag[196] Lost: 0
Done[50]: singer3[131] Lost: 0
Done[51]: nature[999] Lost: 120
Done[52]: traffic[191] Lost: 0
Done[53]: girl[1500] Lost: 6
Done[54]: gymnastics1[567] Lost: 1
Done[55]: fish2[310] Lost: 17
Done[56]: racing[156] Lost: 0
Done[57]: flamingo1[1377] Lost: 11
Done[58]: ants1[325] Lost: 0
Done[59]: handball1[377] Lost: 3
Done[60]: matrix[100] Lost: 5
Done ALL

[goutamgmb]

Hi,

How did you evaluate on vot? Did you use the official VOT toolkit? Also which parameter settings did you use?