can pointpillars model trained by waymo dataset detect bus and truck class ?

[panfengsu]

waymo dataset only have vehicle class,but large vehicles such as bus and truck are labeled vehicle. in hv_pointpillars_secfpn_waymo.py file,only anchor size of car is set as [2.08, 4.73, 1.77].I test the pretrained model and find bus and truck not be detected well. can pointpillars model trained by waymo dataset detect bus and truck class ?

[Tai-Wang]

We only detect vehicle, pedestrian and cyclist here just to follow the official setting of waymo dataset challenge, where only these 3 classes are considered when evaluating the detectors. If you would like to train it for bus and truck detection, please feel free to adjust the classes in the data preprocessing codes, related codes in mmdet3d/datasets/ and configurations for detectors.

[panfengsu]

thanks，I know Waymo datasets only have vehicle, pedestrian and cyclist, but in waymo Vehicle Labeling Specifications, https://github.com/waymo-research/waymo-open-dataset/blob/master/docs/labeling_specifications.md, vehicles other than car also are labeled vehicle class. But I test the pretrained model and find bus which is included vehicle category not be detected well. I want to know why.
I can not adjust the classes in the data preprocessing codes because bus is not separate category in Waymo dataset.

[Tai-Wang]

OK I got it. You did a really good and careful observation. Actually, we did not have a careful investigation into that and just followed the setting of the waymo dataset paper.

In my view it is just related to the anchor setting. For anchor-based detectors, if there exists large variance in terms of size among objects from the same category, a simple way is to give several anchors to this class, like did in the codebase of SECOND. It should also be feasible in mmdet3d, you can make a statistics over sizes of buses and trucks and just have a try.

Another possible reason is that the number of these two sub-classes is much smaller than cars, so the detector will not put much attention on them. The class imbalance problem will be another story.

If there is any interesting conclusion, further discussion and PRs are really welcome.

[panfengsu]

thanks for your detailed explanation. I think it is difficult to handle, on the one hand waymo do not give the number of sub-classes which I will count as much as possible,on the other hand more anchor sizes being set will increase a lot of anchors which is almost negative samples affecting the detection effect.
Car should occupy the majority form high AP of vehicle.I will do more experiments to illustrate and discuss. Thanks.

[tianweiy]

You can do anchor free detector like the CenterPoint model implemented in mmdet3d which should improve results a bit. Here are my results on Waymo.

The results will be much better if we consider the largest 10% and smallest 10% of objects which are presumably bus/truck or motorcycle.

[panfengsu]

Anchor free detector is a good idea and I will test and study CenterPoint carefully, thanks. Category loss of CenterPoint is also focal loss?I do not get good results using focal loss even based anchor methods.But I test your pretrained model of pointpillars and find it can get good result, can you give me some advices?

[tianweiy]

Category loss of CenterPoint is also focal loss

it is a modified focal loss in the cornernet. It is still quite different so it is hard to tell why focal loss doesn't work in your case.

But I test your pretrained model of point pillars and find it can get a good result, can you give me some advices.

the center-based assignment is much simpler and works better for rotated objects / small/large objects. We add some new stuff to the original method and I will update the paper and codebase in a few weeks.

[panfengsu]

It is an outstanding work and I will follow it, thanks.