In the case all class scores are lower than self.cls_threshold_to_reg.

[LHY-HongyangLi]

Dear author @linxuewu ,
This is really a nice work! However, when training "sparse4d_r101_H1" I will always meet a bug at these lines of code.

Sparse4D/projects/mmdet3d_plugin/models/sparse4d_head.py

Lines 257 to 277 in 2154d91

	if self.cls_threshold_to_reg > 0:
	threshold = self.cls_threshold_to_reg
	mask = torch.logical_and(
	mask, cls.max(dim=-1).values.sigmoid() > threshold
	)
	cls = cls.flatten(end_dim=1)
	cls_target = cls_target.flatten(end_dim=1)
	cls_loss = self.loss_cls(cls, cls_target, avg_factor=num_pos)
	mask = mask.reshape(-1)
	reg_weights = reg_weights * reg.new_tensor(self.reg_weights)
	reg_target = reg_target.flatten(end_dim=1)[mask]
	reg = reg.flatten(end_dim=1)[mask]
	reg_weights = reg_weights.flatten(end_dim=1)[mask]
	reg_target = torch.where(
	reg_target.isnan(), reg.new_tensor(0.0), reg_target
	)
	reg_loss = self.loss_reg(
	reg, reg_target, weight=reg_weights, avg_factor=num_pos
	)

Where, the resulting mask is all False, causing an empty "reg_target".

[linxuewu]

If the parameter cls_threshold_to_reg is greater than zero, pretrain model should be used. If pretrain is not added, set this parameter to -1, which has little effect on the effect.

[LHY-HongyangLi]

Yes, I have utilized the pre-trained model "fcos3d.pth" but I still meet the error. Can you train "sparse4d_r101_H1.py" without meeting any error?

[LHY-HongyangLi]

I wrote some code to prevent alerting the bug and print a warning text every time it meets this case. And find that even in the 24th epoch I can still meet this case.

[linxuewu]

I did not encounter this problem. Is your batch_size not large enough? I used 8gpus*1sample_per_gpu.

[linxuewu]

It shouldn’t be considered a bug if all of them are false. Some samples may not have valid object, as long as it does not affect the convergence of loss

[linxuewu]

If you have concerns, just set this parameter to -1. It will not have a significant impact on the model effect

[LHY-HongyangLi]

Yes, you are right. The reason I open this issue is that when I am evaluating the model at the 24th epoch, the mAOE is weird, as shown below. So I'm just wondering if this case causes the weird result.

mAP: 0.3939
mATE: 0.6969
mASE: 0.7060
mAOE: 1.5512
mAVE: 0.7891
mAAE: 0.2022
NDS: 0.3575
Eval time: 119.0s

Per-class results:
Object Class	AP	ATE	ASE	AOE	AVE	AAE
car	0.590	0.488	0.748	1.590	0.924	0.205
truck	0.323	0.744	0.791	1.588	0.837	0.217
bus	0.407	0.792	0.851	1.551	1.694	0.358
trailer	0.174	1.065	0.834	1.617	0.518	0.090
construction_vehicle	0.085	1.105	0.698	1.537	0.180	0.389
pedestrian	0.471	0.632	0.336	1.574	0.448	0.189
motorcycle	0.394	0.655	0.790	1.548	1.232	0.156
bicycle	0.306	0.620	0.807	1.704	0.480	0.015
traffic_cone	0.611	0.414	0.316	nan	nan	nan
barrier	0.577	0.454	0.888	1.251	nan	nan

[linxuewu]

This phenomenon seems to be a problem with the annotation of yaw. It is recommended that you visualize the GT of the training data.

[LHY-HongyangLi]

OK, thank you for your timely reply. I will close this issue.

[bobyang7]

hi，I meet the same

Yes, you are right. The reason I open this issue is that when I am evaluating the model at the 24th epoch, the mAOE is weird, as shown below. So I'm just wondering if this case causes the weird result.

mAP: 0.3939
mATE: 0.6969
mASE: 0.7060
mAOE: 1.5512
mAVE: 0.7891
mAAE: 0.2022
NDS: 0.3575
Eval time: 119.0s

Per-class results:
Object Class	AP	ATE	ASE	AOE	AVE	AAE
car	0.590	0.488	0.748	1.590	0.924	0.205
truck	0.323	0.744	0.791	1.588	0.837	0.217
bus	0.407	0.792	0.851	1.551	1.694	0.358
trailer	0.174	1.065	0.834	1.617	0.518	0.090
construction_vehicle	0.085	1.105	0.698	1.537	0.180	0.389
pedestrian	0.471	0.632	0.336	1.574	0.448	0.189
motorcycle	0.394	0.655	0.790	1.548	1.232	0.156
bicycle	0.306	0.620	0.807	1.704	0.480	0.015
traffic_cone	0.611	0.414	0.316	nan	nan	nan
barrier	0.577	0.454	0.888	1.251	nan	nan

hi，I met the same problem at the 24th epoch，have you solved it？

[linxuewu]

If you are using the code of sparse4dv1 or sparse4dv2, update the version of mmdet3d >= rc1.0.0. @bobyang7