how to generate predictions using fcn8spac?

[Amose-Yao]

Hello,

I am impressed by your great work. And I have some questions about your code.
I used model fcn8spac to train the VOC2012. And after training, I got many pth files. But when I want to generate predictions for VOC test dataset, I can only generate some black images. And I got the normal acc on VOC training data.
And my script is 'CUDA_VISIBLE_DEVICES=3 python -m task_semanticSegmentation.main --data-root data/voc --exp-root exp/voc/fcn8s_pac_but_change_to_pad --load-weights pacnet/exp/voc/fcn8s_pac_but_change_to_pad/weights_epoch_40.pth --test-crop 512 --test-split test --eval pred --model fcn8spac'.
Could you please give me some advices on how to finish that?
Best,
Amose.

[suhangpro]

Since you use --eval pred, predictions should be written as normal images, black indicating background. Not quite sure what went wrong. You can try using --eval raw and inspect the raw predictions directly and see if they make sense, or you can set a breakpoint and see whether it's problem with the image encoding or the prediction itself.

[Amose-Yao]

This is my error message.

Traceback (most recent call last):
File "/home/amose/anaconda3/envs/pacnet/lib/python3.6/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/home/amose/anaconda3/envs/pacnet/lib/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/home/amose/Workspace/pt1/pacnet/task_semanticSegmentation/main.py", line 509, in
main()
File "/home/amose/Workspace/pt1/pacnet/task_semanticSegmentation/main.py", line 321, in main
assert last_epoch in snapshots_found
AssertionError

[suhangpro]

@Amose-Yao This is an error irrelevant to the original issue. If there are weight files in the exp-root during the evaluation, the code is confused about which weight file to use. You can add an --overwrite flag to force using the specified weight file. Sorry that this is not better documented.

[Amose-Yao]

Thanks for your patience. And one more question is how to get the backbone weights like 'fcn8s_from_caffe.pth'? After training I only got some weights named like 'adam_epoch*' and 'weights_epoch*'.

[suhangpro]

Backbone weights are part of the full model so its updated weights are saved in weights_epoch* as well. An exception is CRF models with a frozen backbone (e.g. fcn8sfrozen_crf*), where the backbone is not updated so not stored in the full model.

[Amose-Yao]

Could you please tell me how to train the model fcn8s? I use this script 'python -m task_semanticSegmentation.main --data-root data/voc --exp-root exp/voc/fcn8s --load-weights-backbone fcn8s_from_caffe.pth --train-split train11 --test-split val11_sbd --train-crop 449 --test-crop -1 --model fcn8s --epochs 40 --lr 0.001 --lr-steps 20' and I got many 'weights_epoch*' files in which the parameters are zeros.

[suhangpro]

You should use --load-weights instead of --load-weights-backbone since now fcn8s is the model itself, not just the backbone.

[Amose-Yao]

Hi, everytime I run your code, I would get such error message.
THCudaCheck FAIL file=/pytorch/aten/src/THC/THCGeneral.cpp line=383 error=11 : invalid argument
Is this normal?

[suhangpro]

This is not normal. A possible reason is the incompatible pytorch/Cuda version you are using (the main branch was originally developed for pytorch 0.4 and cuda9). Check out the "th14" branch if you are using the latest pytorch.

[fullsky-1995]

Hi，
I have a question for you when I trainning the fcn8spac. How can me to trainning collectly? And I always get the error such as: RuntimeError: shape '[1, 256, 3, 3, 128, 128]' is invalid for input of size 38340864 at File pac.py, in forward in_mul_k = cols.view(bs, ch, *kernel.shape[2:]) * kernel. Did you meet the same error and how did you solve it?
My enviroment is Cuda 11.1 and pytorch 1.9, thanks very much!