Is val mAP=16.46 reported in paper only trained with 'car' category?

[Senwang98]

@SuperMHP
Hi, thanks to your work.
I want to know if map=16.46 only uses car category during training

I trained my GUPNet by ['Car','Pedestrian','Cyclist'].
Unfortunately, I tried to train three times and could only reach up to 15.5!
So, I doubt how you can train out model with mAP=16.46. (Is the kitti training error of more than 1 point too large?)
Whising for your reply.

[SuperMHP]

Hi:

1. In the original paper, results on the val set are only training with the car category.
2. Mean moderate AP is about 15.5. So It is normal to achieve 15.5 moderate AP. 16.4 is the best one. On the validation set, we run several times and select the best results, which are available for the val set but not suitable for the test set.
3. I think the variance about 1 on the val set is normal. Because the training set of KITTI is small (about 3,000 cases), the results are not quite stable. When you utilize a larger dataset, it is easy to reproduce results because of better stability. For example, training with trainval (about 7000 cases) and evaluating on the test set. It is easy to get higher results than our reports without many times model selections.
4. On the val set, the best result that we train for three categories is our released chkp, about 16.2% moderate. I remember the category used to train will not affect much. The mean difference is that only using car will give higher mean results. But for stability, their variances are similar.

[Senwang98]

@SuperMHP
Great, I got it and thanks for your quick reply!