BiSeNet mean IoU for R18

[ms-krajesh]

Hi!, I am only able to get mean IoU: 70.446% for BiSeNet, when R18 is used as a backbone. I have trained BiSeNet on CityScape leftImg8bit folder (used gtFine folder for GT's) with train input image dimensions 1024x1024.
The achieved results are still little below than the results you mentioned on GitHub repository page (Mean IoU : 74.6). Did you have used network parameters other than mentioned in the config file uploaded on the GitHub repository? Thanks for your time.

[daodaofr]

I get a similar mean IoU 69.688% with R18. I train the model from scratch because the pretrained R18 model is not released. Can you share the pretrained model?

[lxtGH]

Personally I think you should use OHEM loss for training which his paper didn't mention.

[daodaofr]

Personally I think you should use OHEM loss for training which his paper didn't mention.

Thank you but I've already utilized OHEM.

[lxtGH]

Maybe you can try large cropsize

[yu-changqian]

@msc-rajesh Which experiments did you use, cityscapes.bisenet.R18.speed or cityscapes.bisenet.R18? I will re-run it to check the performance.

[daodaofr]

@ycszen I run cityscapes.bisenet.R18 with one GPU. I get mean IoU 69.688%.

[yu-changqian]

@daodaofr I have re-run the cityscapes.bisenet.R18 experiment. The performance is normal. I run this experiment on 4 GPUs. Besides, I think maybe you train from scratch resulting in the performance drop. You can load the official R18 model in Pytorch before I release the pre-trained model.

[ms-krajesh]

@daodaofr : I ran cityscape.bisenet.R18 experiment on 4 GPU's having product name NVidia GeForce GTX 1080 Ti.
I have trained BiSeNet from scratch on CityScape leftImg8bit folder (used "labelTrainIds" instead of "labelIds" from gtFine folder for GT's) with train input image dimensions 1024x1024.
I got mean IoU value 70.446% on validation folder of the CityScape dataset.

[daodaofr]

@ycszen @ms-krajesh
I run the model from official R18 model in Pytorch, and I got 72.753% mean IoU.
I think gap is from the pretrained model.

[ms-krajesh]

@daodaofr : Did you used "labelTrainIds" or "labelIds" for GT's?

[daodaofr]

@ms-krajesh I used labelIds

[chenxiaoyu523]

@daodaofr but the class number in labelids is 33, which is not corresponding with the code, do you have any other processes?

[daodaofr]

@chenxiaoyu523 Yes, I set the label of invalid classes to ignore_index.

[jiaxue-ai]

@ycszen I run your R18.speed and R18 with 4 1080Ti. the accuracies for last epoch models are 74.2 and 75.2. which is a little bit less than your reported accuracy (74.6 and 76.3). Where should be the problem? shall I check the acc for each epoch?

[alexanderfrey]

Can someone upload a pretrained model please ?

[haitaobiyao]

I can't get the right result, and the loss not converge with Bisenet. I don't know why ,can you give me some Suggest

[yu-changqian]

@jiaxue1993 Maybe you can evaluate the models of the last ten epochs.

[yu-changqian]

@alexanderfrey The pre-trained models have released except the Xception39.

[yu-changqian]

@haitaobiyao Could you give more details.

[zhenyouwei]

Hi!, I download the pretrained model(R18)and train model from the link you provided,but I am only able to get mean IoU: 65.2% ,Is there any problem with the parameter setting？