Classification or Segmentation

[MyVanitar]

Hello,

The FCN models which you have offered are classification models or segmentation?
if they are segmentation models, would you please tell me how can I fine-tune it with a pre-trained model?

[JihongJu]

@VanitarNordic They are segmentation models. You can load model weights using model.load_weights and set by_name to be true if you are loading a different model with part of the weights in common.

[MyVanitar]

Thank you very much for your reply.

You can load model weights using model.load_weights and set by_name to be true if you are loading a different model with part of the weights in common.

So if I load the weights and also load the dataset, it will Fine-Tune it, and not do the training from scratch, isn't? Does this applies to the cases when the pre-trained weights is for 21 classes and we want to fine-tune it for example for 2 classes?

I have these questions also:

1. The weights of the Caffe FCN-8s (heavy) and its VOC pre-trained model is belong to the VGG16 model?

2. if I train using VGG19, then I will get better results?

[JihongJu]

@VanitarNordic If you set weights='imagenet' for either FCN_VGG16 or FCN_VGG19, it will load the imagenet pre-trained model accordingly. The other weights will be randomly initialized with 'he_normal'.

1. If you want to load Caffe models, you will need to convert the weights and you can google how to do it.
2. Loading another FCN model with the same architecture but a different number of outputs will cause shape-mismatch problem and you will need to leave out the mismatched layers.
3. I don't know whether VGG16 or VGG19 works better for your problem. It depends.

[MyVanitar]

Okay, thank you very much. I think I got the point. Just these two questions remained:

1. You mean I should download the pre-trained classification ImageNet (either VGG16 or VGG19) as something like "init-model" and start training. may I ask you the download links?

2. Typically a VOC segmentation dataset consists of RGB images and RGB PNG labels and some TXT files which define files' names for the training and validation sets. therefore may I ask you how what is X and Y here?

fcn_vgg19.fit(X_train, y_train, batch_size=1)

[JihongJu]

@VanitarNordic

1. For imagenet models, it will automatically download. See https://github.com/JihongJu/keras-fcn/blob/master/keras_fcn/encoders.py#L71
2. x_train is image(s), a 4D tensor (batch, height, weight, channels), and y_train is segmentation(s), 4D tensor (batch, height, weight, classes)

[MyVanitar]

x_train is image(s), a 4D tensor (batch, height, weight, channels), and y_train is segmentation(s), 4D tensor (batch, height, weight, classes)

Would you please explain this with an Example? both x and y belong to the training set? then when the validation images should be tested?

[JihongJu]

@VanitarNordic fcn_vgg19 is just an instance of keras functional API model. You can use all Model methods, including fit, fit_generator, predict, etc.