checkpoint performance problem

[yypurpose]

Hi! Thank you for doing this inspiring work and sharing the code.

I have a problem with the released checkpoint of EQLv2. I download the provided checkpoint of EQLv2 and evaluate on LVIS1.0 val set using mmdetection. The AP is expected to be 23.7 but got 23.0.
my detailed log is here,

`Evaluating bbox...
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=300 catIds=all] = 0.230
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=300 catIds=all] = 0.396
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=300 catIds=all] = 0.233
Average Precision (AP) @[ IoU=0.50:0.95 | area= s | maxDets=300 catIds=all] = 0.178
Average Precision (AP) @[ IoU=0.50:0.95 | area= m | maxDets=300 catIds=all] = 0.304
Average Precision (AP) @[ IoU=0.50:0.95 | area= l | maxDets=300 catIds=all] = 0.341
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=300 catIds= r] = 0.118
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=300 catIds= c] = 0.211
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=300 catIds= f] = 0.301
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=300 catIds=all] = 0.323
Average Recall (AR) @[ IoU=0.50:0.95 | area= s | maxDets=300 catIds=all] = 0.222
Average Recall (AR) @[ IoU=0.50:0.95 | area= m | maxDets=300 catIds=all] = 0.407
Average Recall (AR) @[ IoU=0.50:0.95 | area= l | maxDets=300 catIds=all] = 0.477

Evaluating segm...
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=300 catIds=all] = 0.230
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=300 catIds=all] = 0.366
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=300 catIds=all] = 0.243
Average Precision (AP) @[ IoU=0.50:0.95 | area= s | maxDets=300 catIds=all] = 0.166
Average Precision (AP) @[ IoU=0.50:0.95 | area= m | maxDets=300 catIds=all] = 0.308
Average Precision (AP) @[ IoU=0.50:0.95 | area= l | maxDets=300 catIds=all] = 0.349
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=300 catIds= r] = 0.140
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=300 catIds= c] = 0.219
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=300 catIds= f] = 0.281
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=300 catIds=all] = 0.321
Average Recall (AR) @[ IoU=0.50:0.95 | area= s | maxDets=300 catIds=all] = 0.210
Average Recall (AR) @[ IoU=0.50:0.95 | area= m | maxDets=300 catIds=all] = 0.408
Average Recall (AR) @[ IoU=0.50:0.95 | area= l | maxDets=300 catIds=all] = 0.483
OrderedDict([('bbox_AP', 0.23), ('bbox_AP50', 0.396), ('bbox_AP75', 0.233), ('bbox_APs', 0.178), ('bbox_APm', 0.304), ('bbox_APl', 0.341), ('bbox_APr', 0.118), ('bbox_APc', 0.211), ('bbox_APf', 0.301), ('bbox_
mAP_copypaste', 'AP:0.230 AP50:0.396 AP75:0.233 APs:0.178 APm:0.304 APl:0.341 APr:0.118 APc:0.211 APf:0.301'), ('segm_AP', 0.23), ('segm_AP50', 0.366), ('segm_AP75', 0.243), ('segm_APs', 0.166), ('segm_APm', 0
.308), ('segm_APl', 0.349), ('segm_APr', 0.14), ('segm_APc', 0.219), ('segm_APf', 0.281), ('segm_mAP_copypaste', 'AP:0.230 AP50:0.366 AP75:0.243 APs:0.166 APm:0.308 APl:0.349 APr:0.140 APc:0.219 APf:0.281')])`

I don't know what's wrong, does anybody get the same problem. I use the standard LVIS1.0 val.

Thanks in advance.

[tztztztztz]

Hi, I test the eqlv2 checkpoint again in this codebase and get a mask AP of 23.7.

The md5 of the checkpoint is: d229ac333700aacd0da5a3946b575eb6
you can check the md5 of the checkpoint first by:

md5sum eqlv2_1x.pth
>> MD5 (eqlv2_1x.pth) = d229ac333700aacd0da5a3946b575eb6

Can you clone this repo and test the checkpoints on it?
I am happy to help if you have any questions.

[yypurpose]

It's okay now using this repo, but I still don't know why （´Д`）

Thanks a lot!