Want to share with you the result on NuScenes validation set here. I have not changed any setting or configuration. Centerpoint input. 2Hz

[BaiLiping]

<style> table {mso-displayed-decimal-separator:"\."; mso-displayed-thousand-separator:"\,";} tr {mso-height-source:auto;} col {mso-width-source:auto;} td {padding-top:1px; padding-right:1px; padding-left:1px; mso-ignore:padding; color:black; font-size:11.0pt; font-weight:400; font-style:normal; text-decoration:none; font-family:等线, sans-serif; mso-font-charset:134; text-align:general; vertical-align:middle; border:none; white-space:nowrap; mso-rotate:0;} </style>

Per-class results:
AMOTA   AMOTP   RECALL  MOTAR   GT      MOTA    MOTP    MT      ML      FAF     TP      FPFN       IDS     FRAG    TID     LGD
bicycle         0.414   1.161   0.478   0.846   1993    0.400   0.160   39      67      10.0    943     1451041    9       8       1.59    2.02
bus             0.721   0.766   0.755   0.957   2112    0.718   0.388   60      13      4.3     1584    68518      10      17      1.40    1.91
car             0.637   0.828   0.700   0.935   58317   0.636   0.241   1714    835     44.6    39656   2564       17504   1157    1019    0.97    1.56
motorcy         0.524   1.013   0.583   0.907   1977    0.516   0.243   36      25      7.7     1124    104824     29      29      1.69    2.35
pedestr         0.688   0.737   0.773   0.873   25423   0.654   0.291   983     292     55.4    19049   2424       5778    596     466     0.63    1.14
trailer         0.384   1.314   0.506   0.711   2425    0.352   0.526   44      56      32.5    1201    3471199    25      31      1.62    2.45
truck           0.475   1.104   0.554   0.822   9650    0.446   0.323   149     192     25.1    5238    9304300    112     120     1.70    2.67
Aggregated results:
AMOTA   0.549
AMOTP   0.989
RECALL  0.621
MOTAR   0.865
GT      14556
MOTA    0.532
MOTP    0.310
MT      3025
ML      1480
FAF     25.7
TP      68795
FP      6582
FN      31164
IDS     1938
FRAG    1690
TID     1.37
LGD     2.01

[ziqipang]

@BaiLiping I have updated my inference files, please refer to README for the dropbox links.

I run SimpleTrack and the performance is correct. In your case, maybe you need to check if you use the correct config file --configs/nu_configs/giou.yaml. For example, my script is python tools/main_nuscenes.py --name cp_sanity --config_path configs/nu_configs/giou.yaml --det_name cp --data_folder /mnt/truenas/scratch/ziqi.pang/datasets/nuscenes/validation_2hz --process 150.

However, I think figuring out the errors will be a lot easier with the inference file I just provided. Good luck!

[BaiLiping]

Got it. Thanks a lot!

[BaiLiping]

i think i know why the result is so different now. The inference file is generated using only 1 lidar sweep right? the inference file i used is generated with 10 aggregated lidar sweeps.

[ziqipang]

@BaiLiping It depends on the detection file you use. The strongest center point uses 10 sweeps, which is my inference file. Maybe yours uses 1 sweep?

[BaiLiping]

mine is 10 sweeps as well. I will have another try with the inference file you provided. Thanks a lot.