Can't reproduce the results of configs/changer/changer_ex_mit-b0_512x512_40k_levircd.py 84.37VS85.29

[dou3516]

My experiment of configs/changer/changer_ex_mit-b0_512x512_40k_levircd.py achieves val/test mIoU 84.83%/84.37%. which significantly below the reported test mIoU 85.29%.

cmds as below:
python tools/train.py configs/changer/changer_ex_mit-b0_512x512_40k_levircd.py --work-dir ./workdir/changer_ex_mit-b0_512x512_40k_levircd
python tools/test.py configs/changer/changer_ex_mit-b0_512x512_40k_levircd.py workdir/changer_mit-b0_levir_workdir/iter_40000.pth

What did I do less？

[likyoo]

Please make sure the pre-trained weights are loaded.

[likyoo]

And I'm not so sure about the stability of the mit-b0 configurations...
If you have new experimental results or conclusions, I sincerely hope you will share them.

[likyoo]

@dou3516
Thanks for pointing out this problem.
I added some notes here.
I guess this may be caused by the low number of parameters in MIT-b0. If you use MIT-b1, this problem may be alleviated.

[dou3516]

And I'm not so sure about the stability of the mit-b0 configurations... If you have new experimental results or conclusions, I sincerely hope you will share them.

experiment IoU results of Changer:

s50: reported/exp 85.51/85.21
MIT-B0: reported/exp 85.29/84.37
MIT-B0: exp 85.04

details as below:

Method	Backbone	Crop Size	Lr schd	Mem (GB)	Precision	Recall	F1-Score	IoU	remark
ChangerEx	s50	512x512	40000	-	93.47	90.95	92.19	85.51	reported
ChangerEx	s50	512x512	40000	-	93.42	90.65	92.02	85.21	exp
ChangerEx	MIT-B0	512x512	40000	-	93.61	90.56	92.06	85.29	reported
ChangerEx	MIT-B0	512x512	40000	-	93.29	89.82	91.52	84.37	exp
ChangerEx	MIT-B2	512x512	40000	-	93.40	90.52	91.94	85.04	exp