How to get class weights with weak labels?

[Anna0509]

Hi @QingyongHu, thanks for your awesome work!
You use class weighted loss while training, where class weights are calculate from full annotations, as shown in the following figure. Have you tried without using class weights? Because we can not get the class weights under the condition of weak supervision. If yes, what about the performance?

[QingyongHu]

Thanks for pointing out this issue!

Yes, we incorrectly calculate the class weights from the full annotations, which actually should calculate based on the sparse annotations. Actually, the class weights under weak supervision are also accessible, just calculate based on the sparse annotated points, while ignoring the unlabeled points. Considering we randomly annotate 0.1% of the points, the class distribution should be similar.

We will re-run the experiments and get back to you.

Thanks!

[Anna0509]

Hi @QingyongHu,

Thanks for your reply. Look forward to your updated results.

[Anna0509]

Hi @QingyongHu,

I'd like to reproduce the results on S3DIS. My script is python main_S3DIS.py --mode test --gpu 0 --test_area 5. But I only got 56.82 mIoU after testing on the full-resolution point clouds. The best snapshot is snap-50001. The detailed result is as follows:

56.82 | 90.79 94.73 77.15 0.00 15.58 51.17 51.09 70.75 80.64 47.09 65.72 44.86 49.07

Can you please check if the provided ConfigS3DIS is correct? Or any other reasons?

Thanks!

[LiXinZhana]

Hi @QingyongHu,

I'd like to reproduce the results on S3DIS. My script is python main_S3DIS.py --mode test --gpu 0 --test_area 5. But I only got 56.82 mIoU after testing on the full-resolution point clouds. The best snapshot is snap-50001. The detailed result is as follows:

56.82 | 90.79 94.73 77.15 0.00 15.58 51.17 51.09 70.75 80.64 47.09 65.72 44.86 49.07

Can you please check if the provided ConfigS3DIS is correct? Or any other reasons?

Thanks!
Hi, my reproduction result is similar to yours, which is very different from the result in the paper, did you solve this problem?
55.79 | 91.98 96.50 75.86 0.01 12.64 47.68 52.13 71.02 81.64 38.78 64.97 43.35 48.68

[jianGao555]

Hi @QingyongHu,
I'd like to reproduce the results on S3DIS. My script is python main_S3DIS.py --mode test --gpu 0 --test_area 5. But I only got 56.82 mIoU after testing on the full-resolution point clouds. The best snapshot is snap-50001. The detailed result is as follows:
56.82 | 90.79 94.73 77.15 0.00 15.58 51.17 51.09 70.75 80.64 47.09 65.72 44.86 49.07
Can you please check if the provided ConfigS3DIS is correct? Or any other reasons?
Thanks!
Hi, my reproduction result is similar to yours, which is very different from the result in the paper, did you solve this problem?
55.79 | 91.98 96.50 75.86 0.01 12.64 47.68 52.13 71.02 81.64 38.78 64.97 43.35 48.68

I'm getting similar results to you.