Loading weights for part of the network

[KlaymenGC]

Hello,

Say I have weights from a trained network and now I want to change the architecture of the network by adding 2 more filters for the last convolutional layer. For the new network, I don't want to train it from scratch again, I want to load the existing weights to the network while initializing the weights for the added 2 filters using other techniques. Is there a way to do it in Keras?

[tboquet]

You could take a look here and in #1728.

[KlaymenGC]

@tboquet Thanks for your reply, I think I don't have trouble adding weights to layers. The thing is, I'm now using a sequential model, say the last convolutional layer is:
Convolution2D(10, 3, 3, activation='relu')
and I've got the trained weights, then I change the last conv layer to:
Convolution2D(12, 3, 3, activation='relu')
and I still want to use the trained weights for the first 10 filters while initializing the weights of the remaining filters using he_normal or something like that.
Do you think it's possible? Could you please elaborate?

[pasky]

According to PR #1796, the canonical way to do this is to build both the old and the new model (you don't need to compile the old one), load weights to the old one, then use the .nodes property to manually transfer the weights to the new model.

[tboquet]

@KlaymenGC oops sorry I didn't understood your question, @pasky is right!

[KlaymenGC]

@pasky thanks for the reply, but it seems that sequential models don't have this node property... any ideas?

[pasky]

I'd try .layers for Sequential models - sorry, it's been long I've been using these.

[fchollet]

For a Sequential model, you can get the layers by index:

weights = model.layers[5].get_weights()

Or by name, if all your layers are named;

layer_dict = dict([(layer.name, layer) for layer in model.layers])
weights = layer_dict['some_name'].get_weights()

[KlaymenGC]

@fchollet yeah but adding weights from
Convolution2D(10, 3, 3, activation='relu')
to
Convolution2D(12, 3, 3, activation='relu') while initializing the weights for the last 2 filters using built-in initialization methods is not possible I guess

[fchollet]

Sure it's possible. But it will require manipulating the weight array in
Numpy-space before calling set_weights(). e.g. array[:-2, :, :, :] =
old_array[:, ;, :, :]

On 2 March 2016 at 14:03, GC notifications@github.com wrote:

@fchollet https://github.com/fchollet yeah but adding weights from
Convolution2D(10, 3, 3, activation='relu')
to
Convolution2D(12, 3, 3, activation='relu') while initializing the weights
for the last 2 filters using built-in initialization methods is not
possible I guess

—
Reply to this email directly or view it on GitHub
#1873 (comment).

[KlaymenGC]

@fchollet nice, thanks for your help!

[bhack]

@fchollet Is it possible instead to add an extra channel to the weights?

[Raazzta]

hi @KlaymenGC, all , could you share your complete code how to combine the weights?
new_weights = pretrained_weights + he_normal

Thanks.