Fast pixelwise analysis throws ValueError for optimizer weight shape

[EricHanLiu]

When running in Rodan with the provided model templates, we get

File "/code/rodan/rodan/jobs/Calvo-classifier/recognition_engine.py", line 47, in process_image_msae
    sae_models.append( load_model(model_paths[id_label]) )
File "/usr/local/lib/python2.7/dist-packages/keras/models.py", line 291, in load_model
     model.optimizer.set_weights(optimizer_weight_values)
File "/usr/local/lib/python2.7/dist-packages/keras/optimizers.py", line 103, in set_weights
    'provided weight shape ' + str(w.shape))
ValueError: Optimizer weight shape (128,) not compatible with provided weight shape (5, 5, 3, 128)`

I've seen a solution that says to install HDFview and open the hdf5 file and delete the optimizer part of the file, which causes the model to load properly. Is this a safe thing to do within the context of this classifier? Will it break any code?

Others have discussed that this is a bug within Keras.

@calvozaragoza any advice?

[calvozaragoza]

It should be safe within the context of classification (the optimizer is only used for training).

However, it seems a version inconsistency problem to me; the fast models were trained before migrating Rodan to docker. Check whether 'theano' is selected as backend in Keras, and also try the different options for the 'image_data_format' setting: https://keras.io/backend/

[EricHanLiu]

Will do, thank you Jorge

[EricHanLiu]

So we confirmed that Theano is the backend for Keras (the config file was the exact same as the instructions in this repository), and there was no image_data_format key so we added it ourselves in the dictionary. Doing so broke docker when we tried to reinstantiate everything, we got the error
Message: No module named tensorflow
and we had to rebuild the image since it crashed every time.
I guess this solution doesn't really work, so we'll try just removing the optimizer part of the models.

[EricHanLiu]

Using this script to remove the optimizer weights for each file, testing now

import h5py
f = h5py.File('model_file.h5', 'r+')
del f['optimizer_weights']
f.close()

[EricHanLiu]

Works with removing the optimizer weights! closing.