LVIS validation issue

[xinqiaozhao]

Hi,
I met an issue when I try to do validation on LVIS v1 dataset. I follow your work and use 4 V100 to do the validation. However, the validation speed is limited to 0.7 img/second ,it is very slow. Is that normal and could you give me a hint about your validation speed? It will be so helpful! Thank you for sharing your work :)

[kemaloksuz]

Hi,

Thank you for your interest.

RS Loss tends to yield higher confidence scores for the predictions than the standard Cross Entropy Loss, hence the NMS score threshold needs to be adjusted accordingly for efficiency. For LVIS dataset, you may be using the default value, which is 0.0001, which causes slower inference speed when the model is trained by RS Loss. Following table presents a comparison of RS-Mask R-CNN and Mask R-CNN regarding this score threshold:

As a result, we recommend setting this threshold to 0.60 for LVIS with RS-Mask R-CNN. We added the LVIS configuration file to the repository yesterday and if you are working with the previous version of the repo, then you should set this score threshold as presented in the following line:

RankSortLoss/configs/ranksort_loss/ranksort_mask_rcnn_r50_fpn_1x_lvis_v1.py

Line 36 in 4afcbc0

test_cfg = dict(rcnn=dict(score_thr=0.60))

[xinqiaozhao]

Thank you so much!!