Inference on the nuscenes test split

[dsx0511]

Hi Haotian @kentang-mit,

the inference on the nuscenes test set is working from my side (refers to this issue) by running the script tools/test.py --format-only. But I want to make sure that I did that correctly:

• I changed line 289 in the file configs/nuscenes/default.yaml to ann_file: ${dataset_root + "nuscenes_infos_test.pkl"} which allows the dataset loads the info pkl file of the test split
• I changed line 277 of mmdet3d/datasets/nuscenes_dataset.py to annos=None
• I commented out lines 207-211 in the file configs/nuscenes/default.yaml that loads annotations (LoadAnnotations3D) in the test pipeline
• I commented out lines 247-259 in the file configs/nuscenes/default.yaml that includes the keys of gt data in Collect3D

By applying these modifications, is it able to produce the results with the same performance on the nuscenes leaderboard? Or do I need some other changes?

Best,
Shuxiao

[kentang-mit]

Hi Shuxiao,

You need to train with data from train+val. Other than that, everything you did is similar to what we did.

Best,
Haotian

[dsx0511]

Hi Haotian, thank you very much for your fast response!

[dikubab]

For the recent version, I think configs/nuscenes/default.yaml lines 250-259 should not be commented out. It works for me with this exception.

[fdy61]

Hi Haotian @kentang-mit,

the inference on the nuscenes test set is working from my side (refers to this issue) by running the script tools/test.py --format-only. But I want to make sure that I did that correctly:

• I changed line 289 in the file configs/nuscenes/default.yaml to ann_file: ${dataset_root + "nuscenes_infos_test.pkl"} which allows the dataset loads the info pkl file of the test split
• I changed line 277 of mmdet3d/datasets/nuscenes_dataset.py to annos=None
• I commented out lines 207-211 in the file configs/nuscenes/default.yaml that loads annotations (LoadAnnotations3D) in the test pipeline
• I commented out lines 247-259 in the file configs/nuscenes/default.yaml that includes the keys of gt data in Collect3D

By applying these modifications, is it able to produce the results with the same performance on the nuscenes leaderboard? Or do I need some other changes?

Best, Shuxiao

[fdy61]

Sorry,it seemes the content in configs/nuscenes/default.yaml and mmdet3d/datasets/nuscenes_dataset.py has been changed.You said changed line 277 of mmdet3d/datasets/nuscenes_dataset.py to annos=None,but the line 277 now is completely unrelated content