miou=0 when running the code on voc2012 dataset

[wwjwy]

I downloaded the voc2012 dataset, and downloaded the weights（pspnet_r101-d8_512x512_20k_voc12aug_20200617_102003-4aef3c9a.pth ） in mmsegmentation, and ran the following code.

python tools/mmseg/train_mmseg.py configs/distill/cwd/cwd_cls_head_pspnet_r101_d8_pspnet_r18_d8_512x1024_voc2012_80k.py --cfg-options algorithm.distiller.teacher.init_cfg.type=Pretrained algorithm.distiller.teacher.init_cfg.checkpoint=pretrained/pspnet_r101-d8_512x512_20k_voc12aug_20200617_102003-4aef3c9a.pth

the validation results are as follows:
+-------------+-------+-------+
| Class | IoU | Acc |
+-------------+-------+-------+
| background | 73.31 | 99.89 |
| aeroplane | 0.0 | 0.0 |
| bicycle | 0.0 | 0.0 |
| bird | 0.0 | 0.0 |
| boat | 0.0 | 0.0 |
| bottle | 0.0 | 0.0 |
| bus | 0.0 | 0.0 |
| car | 0.0 | 0.0 |
| cat | 0.0 | 0.0 |
| chair | 0.0 | 0.0 |
| cow | 0.0 | 0.0 |
| diningtable | 0.0 | 0.0 |
| dog | 0.0 | 0.0 |
| horse | 0.0 | 0.0 |
| motorbike | 0.0 | 0.0 |
| person | 0.32 | 0.33 |
| pottedplant | 0.0 | 0.0 |
| sheep | 0.0 | 0.0 |
| sofa | 0.0 | 0.0 |
| train | 0.0 | 0.0 |
| tvmonitor | 0.0 | 0.0 |
+-------------+-------+-------+
2021-12-24 11:09:10,633 - mmseg - INFO - Summary:
2021-12-24 11:09:10,633 - mmseg - INFO -
+-------+------+------+
| aAcc | mIoU | mAcc |
+-------+------+------+
| 73.25 | 3.51 | 4.77 |
+-------+------+------+

can you help me sove this? Thanks

[pppppM]

Thanks for your attention.
We will solve this issue with you.
Could you upload this config file? @wwjwy

[wwjwy]

thank you for your help!
config.zip

[wwjwy]

The py file in work_dirs is as follow.
work_dirs.zip
s

[wwjwy]

Thanks for your attention. We will solve this issue with you. Could you upload this config file? @wwjwy

When I use this code（https://github.com/pppppM/mmsegmentation-distiller) for testing on voc2012, the result is normal.

[pppppM]

There seem to be two mistakes in your config:

• the norm_cfg type should be SyncBN
• the pascal_voc's num_classes seem to be 21

Could you please check if these two places are consistent with your config in mmsegmentation-distiller? @wwjwy

[wwjwy]

There seem to be two mistakes in your config:

• the norm_cfg type should be SyncBN
• the pascal_voc's num_classes seem to be 21

Could you please check if these two places are consistent with your config in mmsegmentation-distiller? @wwjwy

I changed these two places, but there are still problems。

2021-12-25 23:25:50,995 - mmseg - INFO -
+-------------+-------+-------+
| Class | IoU | Acc |
+-------------+-------+-------+
| background | 73.32 | 100.0 |
| aeroplane | 0.0 | 0.0 |
| bicycle | 0.0 | 0.0 |
| bird | 0.0 | 0.0 |
| boat | 0.0 | 0.0 |
| bottle | 0.0 | 0.0 |
| bus | 0.0 | 0.0 |
| car | 0.0 | 0.0 |
| cat | 0.0 | 0.0 |
| chair | 0.0 | 0.0 |
| cow | 0.0 | 0.0 |
| diningtable | 0.0 | 0.0 |
| dog | 0.0 | 0.0 |
| horse | 0.0 | 0.0 |
| motorbike | 0.0 | 0.0 |
| person | 0.0 | 0.0 |
| pottedplant | 0.0 | 0.0 |
| sheep | 0.0 | 0.0 |
| sofa | 0.0 | 0.0 |
| train | 0.0 | 0.0 |
| tvmonitor | 0.0 | 0.0 |
+-------------+-------+-------+
2021-12-25 23:25:50,995 - mmseg - INFO - Summary:
2021-12-25 23:25:50,996 - mmseg - INFO -
+-------+------+------+
| aAcc | mIoU | mAcc |
+-------+------+------+
| 73.32 | 3.49 | 4.76 |
+-------+------+------+
2021-12-25 23:25:50,998 - mmseg - INFO - Iter(val) [1449] aAcc: 0.7332, mIoU: 0.0349, mAcc: 0.0476, IoU.background: 0.7332, IoU.aeroplane: 0.0000, IoU.bicycle: 0.0000, IoU.bird: 0.0000, IoU.boat: 0.0000, IoU.bottle: 0.0000, IoU.bus: 0.0000, IoU.car: 0.0000, IoU.cat: 0.0000, IoU.chair: 0.0000, IoU.cow: 0.0000, IoU.diningtable: 0.0000, IoU.dog: 0.0000, IoU.horse: 0.0000, IoU.motorbike: 0.0000, IoU.person: 0.0000, IoU.pottedplant: 0.0000, IoU.sheep: 0.0000, IoU.sofa: 0.0000, IoU.train: 0.0000, IoU.tvmonitor: 0.0000, Acc.background: 1.0000, Acc.aeroplane: 0.0000, Acc.bicycle: 0.0000, Acc.bird: 0.0000, Acc.boat: 0.0000, Acc.bottle: 0.0000, Acc.bus: 0.0000, Acc.car: 0.0000, Acc.cat: 0.0000, Acc.chair: 0.0000, Acc.cow: 0.0000, Acc.diningtable: 0.0000, Acc.dog: 0.0000, Acc.horse: 0.0000, Acc.motorbike: 0.0000, Acc.person: 0.0000, Acc.pottedplant: 0.0000, Acc.sheep: 0.0000, Acc.sofa: 0.0000, Acc.train: 0.0000, Acc.tvmonitor: 0.0000
2021-12-25 23:26:01,243 - mmseg - INFO - Iter [100/40000] lr: 9.978e-03, eta: 15:23:31, time: 2.497, data_time: 2.296, memory: 8993, student.decode.loss_ce: 1.3584, student.decode.acc_seg: 55.6429, student.aux.loss_ce: 0.5580, student.aux.acc_seg: 55.7520, distiller

config.zip

[wwjwy]

could you please run the code i just provide on voc2012 dataset and check whether there is something wrong with the code? Thanks!

[pppppM]

Could you upload mmrazor's complete training log and mmseg-distiller's ?
This is very important for me to solve this problem.
@wwjwy

[wwjwy]

Could you upload mmrazor's complete training log and mmseg-distiller's ? This is very important for me to solve this problem. @wwjwy

Today I tried again on my own data set, and the result seems to be normal. The strange thing is that miou seems a bit low, I will upload the complete training log later. Thanks!

[wwjwy]

I trained my own dataset, but i found the miou is low when compared with original model. The training file is attached, can you give me some suggestion on improving model performance? by the way, the r50-pspnet can achieve miou=85.0, the r18-pspnet can only achieve miou=79.33 after training 100000 steps. Thanks!
file.zip

[pppppM]

This seems to be not a problem of mmrazor but a problem of cwd.
I will close this issue for now.
We can discuss it in mmsegmentation-distiller, and you can add the cwd's group in the repo. @wwjwy