Perfomance on val and test set.

[XuyangBai]

Thanks for releasing the wonderful work.

In the readme:

Backbone	mAP	NDS	Download
DETR3D, ResNet101 w/ DCN	34.7	42.2	model | log
above, + CBGS	34.9	43.4	model | log
DETR3D, VoVNet on trainval, evaluation on test set	41.2	47.9	model | log

If I understand correctly, the number of above+CBGS is for the validation set, while the number of DETR3D, VoVNet on trainval is for test set. So I am wondering have you evaluated the results of above + CBGS on the test set? I wonder how much gap it will have for the results on val set and test set

[a1600012888]

Hi, Xuyang.
Your understanding is right.
I don't know the performance of above + CBGS on test set. If I remember correctly, each account can only submit to the test server 3 times a year. We don't have too much test results.

I think you probably can find some numbers in recent papers, where they report the performance of the same model on validation and test set (Both only trained on train set)

[XuyangBai]

Thanks a lot.