Training

[Zumbalamambo]

How do I train on a custom dataset?

[bonlime]

from model import Deeplabv3
deeplab_model = Deeplabv3((512,512,3), num_classes=4, last_activation=True, OS=16)
deeplab_model.load_weights('deeplabv3_weights_tf_dim_ordering_tf_kernels.h5', by_name = True)

after that, you will have a usual Keras model, which you can train using fit, or fit_generator
if you want more detailed description, google any Keras tutorials
There are few important moments with this model:

1. It is important to have a big batch to train BN layers properly, in order to do that you can train with OS 16 and inference with OS 8
2. you can also freeze first 356 layers and train only decoder with big batch size first, then unfreeze all layers and train with smaller batch size

[Soulempty]

Can you give some advice on how to train on a new image datasets?I want to finetune with the pretrained model (cityscapes).

[kli017]

I try to initial the model by
deeplab_model = Deeplabv3((512,512,3), num_classes=4, last_activation=True, OS=16)
by the get below error
TypeError: Deeplabv3() got an unexpected keyword argument 'num_classes'

[bonlime]

@kli017 interface changed a little bit. Look at the new one in README

[kli017]

@bonlime Thanks! The new one in README has a parameter named 'weighs' , however in the model.py the parameter is 'weights'. I tried both of them but still get error

deeplab_model = Deeplabv3((960,1440,3), classes=4, weighs = None, OS=16)
TypeError: Deeplabv3() got an unexpected keyword argument 'weighs'

[bonlime]

Sorry, it was typo. Correct version is "weights", you should have problems with it

[kli017]

Hello, I also tried 'weights' with both None and 'pascal_voc'. But still got TypeError...
TypeError: Deeplabv3() got multiple values for keyword argument 'weights'

[raghavmallick]

@kli017 Try using the arguments in the following format:

Deeplabv3(weights='pascal_voc', input_tensor=None, input_shape=(512, 512, 3), classes=21, OS=16)

This worked for me

[bonlime]

@kli017 the reason why you get TypeError is because you pass input_shape as first argument without explicitly naming it. all parameters you pass into this model have to be named

[bhack]

@bonlime What is the memory requirements to train with the standard input and batch size? The full trainabile end to end (encoder+decoder) model seems quite big to fit in memory.

[bhack]

Could make sense to load and freeze low-level only weights? https://github.com/tensorflow/models/blob/master/research/deeplab/g3doc/model_zoo.md#model-details-2

[bonlime]

@bhack you're right, the model is really big. On my 1080ti it is possible to train only with a batch size of 6 images.
Yeah, it can make sense to freeze low-level features. Adding only ImageNet pre-trained xception is already in my checklist

[wenouyang]

@bhack, the link you shared is the tensorflow-based weights, how can I load it in the keras model, please advise. Thanks

https://github.com/tensorflow/models/blob/master/research/deeplab/g3doc/model_zoo.md#model-details-2

[bhack]

You need to extend extract_weights.py and load_weights.py in this repository.

[Taylor-Rose]

Do I still have to fit after load_weights? or I can directly fit model after model=Deeplabv3()?

[bonlime]

@Taylor-Rose you don't need to do that anymore. Now weights are loaded if you pass "weights" argument

[Taylor-Rose]

@bonlime I did some code like this:
model = Deeplabv3() model.summary() model.fit(train_data,label_data, epochs etc.)
I found that there is no function like 'sigmoid' or 'softmax' on the last layer of the model. In the U-net model I found the function like 'sigmoid' or 'softmax'. What I want to ask is that can I use model.fit() directly after model=Deeplabv3() without any other process. The second I want to ask is that the input data's formation. Does train_data's formation like (512,512,3) with 3 dimension and label_data's formation also like (512,512,3) or should do some preprocess to (class_num, 512,512,3) or other formation?

[bhack]

@bonlime Are the extract_weights/load_weights recovering the BN mean and variance? Check keras-team/keras#3423 (comment).

[bhack]

@bonlime See also this long topic keras-team/keras#9965

[bonlime]

@bhack sorry for the long answer. Both moving mean and variance are recovered correctly.
btw. Thanks for pointing out the problem with frozen BN. What kind of work around this problem do you personally use?

[JackMing1986]

How long can I finish training?

[ChengxiHAN]

Have u guys trained new datasets and fine tuned successfully？plz tell me，I wanna try......

[margokhokhlova]

Hello everyone! Can someone give me an advice, please? I want to try the model on a binary segmentation, shall I add a sigmoid activation to the last model output?

[pluniak]

@ChengxiHAN @margokhokhlova Did you read issue #56? Yes sigmoid activation on the last layer. I have fine-tuned successfully, but freezing BN-layers while finetuning might cause problems.

[margokhokhlova]

@pluniak Thank you! Reading it.

[Chenyang-Qu]

@bonlime I did some code like this:
model = Deeplabv3() model.summary() model.fit(train_data,label_data, epochs etc.)
I found that there is no function like 'sigmoid' or 'softmax' on the last layer of the model. In the U-net model I found the function like 'sigmoid' or 'softmax'. What I want to ask is that can I use model.fit() directly after model=Deeplabv3() without any other process. The second I want to ask is that the input data's formation. Does train_data's formation like (512,512,3) with 3 dimension and label_data's formation also like (512,512,3) or should do some preprocess to (class_num, 512,512,3) or other formation?

Hollow.Have you successfully run this model?I got an errror :AttributeError: 'ModelCheckpoint' object has no attribute 'on_train_batch_begin'
I used fit_generator to get train data and val data.

[Zumbalamambo]

@Cris-Qu Yes it works perfectly at the moment

[Zumbalamambo]

@ChengxiHAN It works fine once we train with the custom dataset. The accuracy was fair enough