How to training? #52
Comments
|
Unfortunately, not quite, because:
Hope this answers your question. |
|
Hi, I use trian.prototxt add the data layer and loss layer ,when I set crop=347, the error is cudaSuccess out of memory, when I set crop=147, error is number of labels must match number of predictions, could you help me with that? BTW, i also confuse how to make sure source image size is same as label image size? name: "pspnet101_VOC2012" layer { layer { |
|
@landiaokafeiyan Does your training prototxt work? How about the solver.txt? |
|
我上面的问题还没有解决，所以我不确定是不是work |
|
@xhsoldier 你可以加我扣扣我们讨论一下 307821808 |
|
after reading the paper, I found the loss function is two softmax loss function addup together with the weight of 0.4. |
|
@xhsoldier Maybe you could set "iter_size: n" parameter in the solver.prototxt to make your actual batchsize be equal to n * batchsize, avoiding the insufficient GPU memory issue of training with a single GPU. |
|
@xhsoldier |
|
Here is the training proto, I can train by using these protos. https://github.com/SoonminHwang/caffe-segmentation/tree/master/pspnet/models |
|
@xhsoldier The weird thing is that no matter what learning rate I set, I always end up with this same loss value With your given prototxt, I have set this hyperparameter: Let me know if you find something wrong here. @qizhuli Hi buddy, Thanks in advance. |
|
you need an initial model, just fine-tune and you will get right result, do not train from scratch. |
|
@xhsoldier It worked. Thank you for showing me the right direction. I am pretty much following your caffe-segmentation code. Its very well organized and easy to understand. Great work. One last question, The loss seems to fluctuate little bit, but per class accuracy is pretty much darn same for all iteration. So the questions is, how can I get the stable per class accuracy in my training. |
|
@landiaokafeiyan Can you share for me training script? |
|
@bhadresh74 What init weight did you use in training? |
|
@ThienAnh i am on a holiday now. maybe i will send you next monday. i just changed the input of the test prototxt |
|
@Dasona I have used VOC 2012 pre-trained model as my initial weights. |
|
@landiaokafeiyan Thank you so much. Have enjoy holiday! |
|
@landiaokafeiyan Are you come back from holiday? |
|
@ThienAnh hi can you give me your email，so i can send you all materials |
|
@landiaokafeiyan My Email: nguyenthienanh@gmail.com |
|
@landiaokafeiyan Can you forward me as well? |
|
HI @landiaokafeiyan . Train net output #0: accuracy = 0.768128 |
|
@ThienAnh Hi, Maybe you must reduce your learning rate. Regards, Liangyan Li |
|
@landiaokafeiyan |
|
@ThienAnh liangyan |
|
HI @xhsoldier, @landiaokafeiyan Why we need initial model? Can i using pspnet101_VOC2012.caffemodel for init model and train with my data ? (I have 21 labels: ring, clothers, box, cap, hat......It is difference with VOC2012 ) Thanks so much |
|
@ThienAnh You can use pspnet101_VOC2012.caffemodel for tuning or training. |
|
@xhsoldier Thank you so much! |
|
@xhsoldier @ThienAnh @landiaokafeiyan Hi，I am trying to train the network using my own annotated dataset. After i read the prototxt above, I am wondering the context in your pascal_voc_train_aug.txt file and where to put the images and segmentation annotation. I know in PASCAL_VOC the images are in the Imagesets folder and the segmentation annotations are in the SegmentationClass folder. So, how to organize my own dataset and what is in the pascal_voc_train_aug.txt file? Thank you!!! |
|
@fbi0817 You can search and download SegmentationClassAug for see format of label images. In this: source: "/home/adminpxz/PSPNet/PSPNet/splits/pascal_voc_train_aug.txt" is list of training images(.jpg) and label images (.png). Label images is png 8 bit. If you open once image in SegmentationClassAug with msPaint, you can see that areas of object label will have value from 1-20 ( 20 classes). |
|
@ThienAnh Thank you for your quick reply. I will have a try. |
|
@bhadresh74 @ThienAnh @xhsoldier Using the training prototxt file from https://github.com/SoonminHwang/caffe-segmentation/tree/master/pspnet/models Thanks, F0125 16:54:05.818322 25 math_functions.cu:121] Check failed: status == CUBLAS_STATUS_SUCCESS (11 vs. 0) CUBLAS_STATUS_MAPPING_ERROR |
|
@ThienAnh EDIT: I tried removing the label_type: PIXEL and it at least doesn't get that error. Now I need to figure out if the training is correct. |
|
@xhsoldier @ThienAnh @landiaokafeiyan |
|
@ThienAnh |
|
I've implemented sync batch normalization in pure tensorflow, which makes possible to train and reproduce the performance of PSPNet: https://github.com/holyseven/PSPNet-TF-Reproduce. Have a look at the code if you are interested. |
|
@holyseven Thank you for your code! |
|
@ThienAnh Thank you for sharing your train.prototxt file. I tried with your settings but unable to fine tune the pre-trained PSPNet. Have you had success with this settings. |
|
Since there are quite a few people on this thread I thought I would mention that I have also been able to train PSPNet50 (50 layer PSPNet) in Tensorflow from ResNet50 weights. Instead of implementing Batch Norm synchronization across GPUs, I was able to fit the entire model on a single Titan Xp using gradient checkpointing. I trained with a batch size of 8 and got the same accuracy as described in the ICNet paper. The repository for the project can be here: https://github.com/oandrienko/fast-semantic-segmentation/. You can find a link to the trained PSPNet model here. There is also a usage guide for training PSPNet50 here. |
|
@ThienAnh hi, i was also try to training PSPNet lately and i doing everything ok, but when i use the eval_all.m to test my caffemodel, the whole predict image is wrong. i just want to ask your predict image is fine? i want to figure out which step i was wrong, thank you very much!!! |
Using this:
PSPNet/evaluation/prototxt/pspnet101_VOC2012_473.prototxt
change the data layer to the training data?
The text was updated successfully, but these errors were encountered: