AssertionError: assert unachieved_thresholds + duplicate_thresholds + len(thresh_metrics) == self.num_thresholds

[s1mpleee]

Running:

python tools/nusc_tracking/pub_test.py --work_dir work_dirs/tracking --checkpoint ckpt/infos_val_10sweeps_withvelo_filter_True.json

Got:

Loading NuScenes tables for version v1.0-trainval...
23 category,
8 attribute,
4 visibility,
64386 instance,
12 sensor,
10200 calibrated_sensor,
2631083 ego_pose,
68 log,
850 scene,
34149 sample,
2631083 sample_data,
1166187 sample_annotation,
4 map,
Done loading in 26.2 seconds.
======
Reverse indexing ...
Done reverse indexing in 7.2 seconds.
======
Deploy OK
Use hungarian: False
Begin Tracking

The speed is 499.219186836908 FPS
======
Loading NuScenes tables for version v1.0-trainval...
23 category,
8 attribute,
4 visibility,
64386 instance,
12 sensor,
10200 calibrated_sensor,
2631083 ego_pose,
68 log,
850 scene,
34149 sample,
2631083 sample_data,
1166187 sample_annotation,
4 map,
Done loading in 25.9 seconds.
======
Reverse indexing ...
Done reverse indexing in 7.3 seconds.
======
Initializing nuScenes tracking evaluation
Loaded results from work_dirs/tracking/tracking_result.json. Found detections for 6019 samples.
Loading annotations for val split from nuScenes version: v1.0-trainval
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 6019/6019 [00:06<00:00, 975.19it/s]
Loaded ground truth annotations for 6019 samples.
Filtering tracks
=> Original number of boxes: 507598
=> After distance based filtering: 378345
=> After LIDAR points based filtering: 378345
=> After bike rack filtering: 377683
Filtering ground truth tracks
=> Original number of boxes: 142261
=> After distance based filtering: 103564
=> After LIDAR points based filtering: 93885
=> After bike rack filtering: 93875
Accumulating metric data...
Computing metrics for class bicycle...

Computed thresholds                                                                                                                                                                                                            

                MOTAR   MOTP    Recall  Frames  GT      GT-Mtch GT-Miss GT-IDS  Pred    Pred-TP Pred-FP Pred-IDS                                                                                                               
thr_0.1061      0.000   0.345   0.886   6019    1993    1717    228     48      90463   1717    88698   48

                MOTAR   MOTP    Recall  Frames  GT      GT-Mtch GT-Miss GT-IDS  Pred    Pred-TP Pred-FP Pred-IDS                                                                                                               
thr_0.1517      0.000   0.339   0.869   6008    1993    1698    262     33      47831   1698    46100   33

                MOTAR   MOTP    Recall  Frames  GT      GT-Mtch GT-Miss GT-IDS  Pred    Pred-TP Pred-FP Pred-IDS                                                                                                               
thr_0.1623      0.000   0.343   0.857   5995    1993    1678    285     30      39127   1678    37419   30

                MOTAR   MOTP    Recall  Frames  GT      GT-Mtch GT-Miss GT-IDS  Pred    Pred-TP Pred-FP Pred-IDS                                                                                                               
thr_0.1943      0.000   0.344   0.832   5776    1993    1634    335     24      21386   1634    19728   24

                MOTAR   MOTP    Recall  Frames  GT      GT-Mtch GT-Miss GT-IDS  Pred    Pred-TP Pred-FP Pred-IDS                                                                                                               
thr_0.2077      0.000   0.335   0.806   5558    1993    1586    387     20      16451   1586    14845   20

                MOTAR   MOTP    Recall  Frames  GT      GT-Mtch GT-Miss GT-IDS  Pred    Pred-TP Pred-FP Pred-IDS                                                                                                               
thr_0.2206      0.000   0.327   0.784   5266    1993    1545    430     18      13193   1545    11630   18

                MOTAR   MOTP    Recall  Frames  GT      GT-Mtch GT-Miss GT-IDS  Pred    Pred-TP Pred-FP Pred-IDS                                                                                                               
thr_0.2489      0.000   0.310   0.760   4420    1993    1499    479     15      8516    1499    7002    15

                MOTAR   MOTP    Recall  Frames  GT      GT-Mtch GT-Miss GT-IDS  Pred    Pred-TP Pred-FP Pred-IDS                                                                                                               
thr_0.2682      0.000   0.305   0.729   3914    1993    1441    540     12      6515    1441    5062    12

                MOTAR   MOTP    Recall  Frames  GT      GT-Mtch GT-Miss GT-IDS  Pred    Pred-TP Pred-FP Pred-IDS                                                                                                               
thr_0.2961      0.000   0.288   0.702   3143    1993    1391    593     9       4441    1391    3041    9

                MOTAR   MOTP    Recall  Frames  GT      GT-Mtch GT-Miss GT-IDS  Pred    Pred-TP Pred-FP Pred-IDS                                                                                                               
thr_0.3237      0.000   0.285   0.683   2549    1993    1355    631     7       3303    1355    1941    7

                MOTAR   MOTP    Recall  Frames  GT      GT-Mtch GT-Miss GT-IDS  Pred    Pred-TP Pred-FP Pred-IDS                                                                                                               
thr_0.3778      0.323   0.282   0.655   1932    1993    1302    688     3       2187    1302    882     3

                MOTAR   MOTP    Recall  Frames  GT      GT-Mtch GT-Miss GT-IDS  Pred    Pred-TP Pred-FP Pred-IDS                                                                                                               
thr_0.3938      0.462   0.279   0.636   1795    1993    1266    725     2       1949    1266    681     2

                MOTAR   MOTP    Recall  Frames  GT      GT-Mtch GT-Miss GT-IDS  Pred    Pred-TP Pred-FP Pred-IDS                                                                                                               
thr_0.4033      0.493   0.279   0.614   1771    1993    1222    769     2       1843    1222    619     2

                MOTAR   MOTP    Recall  Frames  GT      GT-Mtch GT-Miss GT-IDS  Pred    Pred-TP Pred-FP Pred-IDS                                                                                                               
thr_0.4222      0.570   0.281   0.582   1697    1993    1158    834     1       1657    1158    498     1

                MOTAR   MOTP    Recall  Frames  GT      GT-Mtch GT-Miss GT-IDS  Pred    Pred-TP Pred-FP Pred-IDS                                                                                                               
thr_0.4251      0.570   0.270   0.561   1684    1993    1118    874     1       1600    1118    481     1

                MOTAR   MOTP    Recall  Frames  GT      GT-Mtch GT-Miss GT-IDS  Pred    Pred-TP Pred-FP Pred-IDS                                                                                                               
thr_0.4410      0.618   0.268   0.546   1646    1993    1088    905     0       1504    1088    416     0

                MOTAR   MOTP    Recall  Frames  GT      GT-Mtch GT-Miss GT-IDS  Pred    Pred-TP Pred-FP Pred-IDS                                                                                                               
thr_0.4791      0.750   0.274   0.514   1518    1993    1024    969     0       1280    1024    256     0

                MOTAR   MOTP    Recall  Frames  GT      GT-Mtch GT-Miss GT-IDS  Pred    Pred-TP Pred-FP Pred-IDS                                                                                                               
thr_0.4946      0.772   0.266   0.492   1500    1993    980     1013    0       1203    980     223     0

                MOTAR   MOTP    Recall  Frames  GT      GT-Mtch GT-Miss GT-IDS  Pred    Pred-TP Pred-FP Pred-IDS                                                                                                               
thr_0.5247      0.812   0.267   0.461   1459    1993    918     1075    0       1091    918     173     0

                MOTAR   MOTP    Recall  Frames  GT      GT-Mtch GT-Miss GT-IDS  Pred    Pred-TP Pred-FP Pred-IDS                                                                                                               
thr_0.5385      0.847   0.272   0.437   1432    1993    870     1123    0       1003    870     133     0

                MOTAR   MOTP    Recall  Frames  GT      GT-Mtch GT-Miss GT-IDS  Pred    Pred-TP Pred-FP Pred-IDS                                                                                                               
thr_0.5446      0.853   0.276   0.419   1426    1993    836     1157    0       959     836     123     0

                MOTAR   MOTP    Recall  Frames  GT      GT-Mtch GT-Miss GT-IDS  Pred    Pred-TP Pred-FP Pred-IDS                                                                                                               
thr_0.5746      0.892   0.284   0.384   1398    1993    766     1227    0       849     766     83      0

                MOTAR   MOTP    Recall  Frames  GT      GT-Mtch GT-Miss GT-IDS  Pred    Pred-TP Pred-FP Pred-IDS                                                                                                               
thr_0.5824      0.895   0.234   0.367   1392    1993    731     1262    0       808     731     77      0

                MOTAR   MOTP    Recall  Frames  GT      GT-Mtch GT-Miss GT-IDS  Pred    Pred-TP Pred-FP Pred-IDS                                                                                                               
thr_0.5951      0.907   0.232   0.339   1385    1993    676     1317    0       739     676     63      0

                MOTAR   MOTP    Recall  Frames  GT      GT-Mtch GT-Miss GT-IDS  Pred    Pred-TP Pred-FP Pred-IDS                                                                                                               
thr_0.6033      0.909   0.238   0.314   1383    1993    626     1367    0       683     626     57      0

                MOTAR   MOTP    Recall  Frames  GT      GT-Mtch GT-Miss GT-IDS  Pred    Pred-TP Pred-FP Pred-IDS                                                                                                               
thr_0.6149      0.913   0.239   0.295   1377    1993    588     1405    0       639     588     51      0

                MOTAR   MOTP    Recall  Frames  GT      GT-Mtch GT-Miss GT-IDS  Pred    Pred-TP Pred-FP Pred-IDS                                                                                                               
thr_0.6236      0.917   0.244   0.271   1374    1993    541     1452    0       586     541     45      0

                MOTAR   MOTP    Recall  Frames  GT      GT-Mtch GT-Miss GT-IDS  Pred    Pred-TP Pred-FP Pred-IDS                                                                                                               
thr_0.6369      0.926   0.249   0.251   1368    1993    500     1493    0       537     500     37      0

                MOTAR   MOTP    Recall  Frames  GT      GT-Mtch GT-Miss GT-IDS  Pred    Pred-TP Pred-FP Pred-IDS                                                                                                               
thr_0.6439      0.933   0.255   0.225   1363    1993    449     1544    0       479     449     30      0

                MOTAR   MOTP    Recall  Frames  GT      GT-Mtch GT-Miss GT-IDS  Pred    Pred-TP Pred-FP Pred-IDS                                                                                                               
thr_0.6590      0.942   0.230   0.200   1356    1993    398     1595    0       421     398     23      0

                MOTAR   MOTP    Recall  Frames  GT      GT-Mtch GT-Miss GT-IDS  Pred    Pred-TP Pred-FP Pred-IDS                                                                                                               
thr_0.6607      0.940   0.234   0.192   1356    1993    383     1610    0       406     383     23      0

                MOTAR   MOTP    Recall  Frames  GT      GT-Mtch GT-Miss GT-IDS  Pred    Pred-TP Pred-FP Pred-IDS                                                                                                               
thr_0.6642      0.929   0.146   0.156   1355    1993    310     1683    0       332     310     22      0

                MOTAR   MOTP    Recall  Frames  GT      GT-Mtch GT-Miss GT-IDS  Pred    Pred-TP Pred-FP Pred-IDS                                                                                                               
thr_0.6793      0.917   0.147   0.126   1354    1993    252     1741    0       273     252     21      0

                MOTAR   MOTP    Recall  Frames  GT      GT-Mtch GT-Miss GT-IDS  Pred    Pred-TP Pred-FP Pred-IDS                                                                                                               
thr_0.6912      0.936   0.137   0.102   1347    1993    203     1790    0       216     203     13      0

Traceback (most recent call last):
  File "tools/nusc_tracking/pub_test.py", line 192, in <module>
    eval_tracking()
  File "tools/nusc_tracking/pub_test.py", line 160, in eval_tracking
    args.root
  File "tools/nusc_tracking/pub_test.py", line 178, in eval
    metrics_summary = nusc_eval.main()
  File "/media/adas/ubuntu4/project/nuscenes-devkit/python-sdk/nuscenes/eval/tracking/evaluate.py", line 205, in main
    metrics, metric_data_list = self.evaluate()
  File "/media/adas/ubuntu4/project/nuscenes-devkit/python-sdk/nuscenes/eval/tracking/evaluate.py", line 135, in evaluate
    accumulate_class(class_name)
  File "/media/adas/ubuntu4/project/nuscenes-devkit/python-sdk/nuscenes/eval/tracking/evaluate.py", line 131, in accumulate_class
    curr_md = curr_ev.accumulate()
  File "/media/adas/ubuntu4/project/nuscenes-devkit/python-sdk/nuscenes/eval/tracking/algo.py", line 156, in accumulate
    assert unachieved_thresholds + duplicate_thresholds + len(thresh_metrics) == self.num_thresholds
AssertionError

[DeclK]

I met the same problem, did you solve this?

[s1mpleee]

try pip install numpy==1.19.2

[DeclK]

Thanks for the quick reply, it works perfectly!