Some question about pretrained model

[MaxShi007]

Thank you for your nice work，i have some question about pretrained model
I downloaded the model “10_scans.ckpt” and try to eval it on semantic kitti dataset use semantic kitti API , but I can't get the results in the paper.
this is the result in paper: w/o bf 74.3
this is my result : w/o bf 52.2
Did I miss something？

[benemer]

Hi Max Shi,

Thanks for your interest in our work. The reason for the different result on the SemanticKITTI validation sequence 08 is that we used the SuMa poses without loop closing for validation. The poses with loop closing lead to a lot of false positives for some scans due to a drift induced by the loop closing in sequence 08.

You can download the poses without loop closing here. Just put them in the same folder as the poses.txt file and provide the filename when running the inference script by passing the -poses flag:

python scripts/predict_confidences.py -w path/to/model.ckpt -poses name_pose_file.txt -seq 8

Please let me know if anything is unclear.

Best regards
Benedikt

[MaxShi007]

Hi Max Shi,

Thanks for your interest in our work. The reason for the different result on the SemanticKITTI validation sequence 08 is that we used the SuMa poses without loop closing for validation. The poses with loop closing lead to a lot of false positives for some scans due to a drift induced by the loop closing in sequence 08.

You can download the poses without loop closing here. Just put them in the same folder as the poses.txt file and provide the filename when running the inference script by passing the -poses flag:

python scripts/predict_confidences.py -w path/to/model.ckpt -poses name_pose_file.txt -seq 8

Please let me know if anything is unclear.

Best regards Benedikt

it work! thank you so much

[MaxShi007]

Hi，benemer

I saw your score in the CodaLab, You are the champion and 0.8 is pretty good result. Congratulation！

I guess you may have used SuMa poses without loop in sequence11-21，If you use SuMa poses without loop, can you give me a copy？I want to make little attempt and see the effect.

[benemer]

Hi @MaxShi007,

Thank you! We achieved this result by training our model using 10 scans with additional training data. The data is taken from the KITTI Road sequences and the labels can be found here. We added all labeled sequences to the training set.

If you want to try the SuMa poses without loop closing, you can download them here.

Best regards
Benedikt

[MaxShi007]

Thank you for providing this information about KITTI Road , I will try it.

Here is only have the SuMa poses without loop closing of the trainset（sequence 00-07，09-10） and the validset（sequence 08）. Do you have the SuMa poses without loop closing of the testset（sequence 11-21）？

[benemer]

Ah sorry, you are right. Here are the SuMa poses on the test set without loop closing:

suma_noloop_testset.zip

If I remember correctly, there were a lot of false positives on sequence 20 due to some inconsistent poses (even without loop closing).

Best,
Benedikt