load_model() with custom layers, and custom layers in general

[keunwoochoi]

1. What should I do to make my custom layers available to be loaded using load_model()?

I used my custom layers in this repo both Spectrogram and Melspectrogram didn't work for load_model().

Error message:

>>> keras.models.load_model('model_best.hdf5')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/gnu/anaconda/lib/python2.7/site-packages/keras/models.py", line 140, in load_model
    model = model_from_config(model_config, custom_objects=custom_objects)
  File "/Users/gnu/anaconda/lib/python2.7/site-packages/keras/models.py", line 189, in model_from_config
    return layer_from_config(config, custom_objects=custom_objects)
  File "/Users/gnu/anaconda/lib/python2.7/site-packages/keras/utils/layer_utils.py", line 34, in layer_from_config
    return layer_class.from_config(config['config'])
  File "/Users/gnu/anaconda/lib/python2.7/site-packages/keras/models.py", line 1055, in from_config
    layer = get_or_create_layer(first_layer)
  File "/Users/gnu/anaconda/lib/python2.7/site-packages/keras/models.py", line 1039, in get_or_create_layer
    layer = layer_from_config(layer_data)
  File "/Users/gnu/anaconda/lib/python2.7/site-packages/keras/utils/layer_utils.py", line 33, in layer_from_config
    instantiate=False)
  File "/Users/gnu/anaconda/lib/python2.7/site-packages/keras/utils/generic_utils.py", line 16, in get_from_module
    str(identifier))
Exception: Invalid layer: Melspectrogram
>>>

2. In the same manner, I think we should update layer customising documentation. My questions would be

• if I have to use self.add_weight() method? or just appending them in self.trainable_weights is fine?
• What do I have to do with get_config() method? When and how are they used?
• How self.built is used?

[bstriner]

Use custom_objects to pass a dictionary to load_model. It isn't documented under load_model but it's documented under layer_from_config. This way you can load custom layers.

https://github.com/fchollet/keras/blob/master/keras/utils/layer_utils.py

Make sure to implement get_config() in your custom layer, it is used to save the model correctly.

If you have a lot of issues with load_model, save_weights and load_weights can be more reliable.

Cheers,
Ben

[keunwoochoi]

Thanks. Luckily I could use load_weights() for the moment, but want to fix the problem.
I don't fully understand how custom_objects is related exactly. Just checked out load_model() is defined as

def load_model(filepath, custom_objects=None):

so I guess I'm supposed to send a dictionary when I use load_model(). But what kinds of dictionary it should be? How it is related to my get_config() implementation?

I already implemented get_config() as

    def get_config(self):
        config = {'n_dft': self.n_dft,
                  'n_filter': self.n_filter,
                  'trainable_kernel': self.trainable_kernel,
                  'n_hop': self.n_hop,
                  'border_mode': self.border_mode,
                  'power': self.power,
                  'return_decibel_spectrogram': self.return_decibel_spectrogram}
        base_config = super(Spectrogram, self).get_config()
        return dict(list(base_config.items()) + list(config.items()))

. I just added the attributes of the custom layer without understand what there should be.

[bstriner]

Models.py saves the layer configuration and the class name.

for layer in self.layers[0].layers:
                layer_config = {'class_name': layer.__class__.__name__,
                                'config': layer.get_config()}
                layers.append(layer_config)

layer_from_config instantiates the class given the name.

layer_class = get_from_module(class_name, globals(), 'layer',
                                      instantiate=False)

Custom objects are added to the globals.

  if custom_objects:
       for cls_key in custom_objects:
           globals()[cls_key] = custom_objects[cls_key]

So, if you're crashing on get_from_module, pass in a dictionary from the string of the class name to the actual class so get_from_module can find your class. {"MyClass":MyClass}

In terms of what to put into get_config, that same information is sent to your constructor. So the keywords in your get_config should match the kwargs of your constructor.

def from_config(cls, config):
      '''This method is the reverse of get_config,
      capable of instantiating the same layer from the config
      dictionary. It does not handle layer connectivity
      (handled by Container), nor weights (handled by `set_weights`).
      # Arguments
          config: A Python dictionary, typically the
              output of get_config.
      '''
      return cls(**config)

https://github.com/fchollet/keras/blob/master/keras/utils/layer_utils.py

[keunwoochoi]

Perfect. Thanks a lot.

[NumesSanguis]

For the people landing here by a Google search, the code we should use:

model = keras.models.load_model('temp_model.h5',
              custom_objects={'Melspectrogram':kapre.time_frequency.Melspectrogram})

as can be found on kapre's Github page: https://github.com/keunwoochoi/kapre

[saritanavuluru]

My model has a custom layer - SpatialTransformer which takes in 3 arguments.

model.add(SpatialTransformer(localization_net=locnet,
input_shape=(32,32,1),output_size=(32,32)))

Going by your last message, I added the 'custom_objects' attrib to the load_model call.
model_deep= load_model('deep_cnn.h5',custom_objects={'SpatialTransformer':SpatialTransformer})
I get the following error -
TypeError: init() missing 2 required positional arguments: 'localization_net' and 'output_size'

from above posts -
layer_from_config instantiates the class given the name.
layer_class = get_from_module(class_name, globals(), 'layer',
instantiate=False)

How are arguments passed ?

[NumesSanguis]

As far as I understand, custom layers only support loading if the author(s) implemented it. For example, the seq2seq model also doesn't support it, as can be found here: farizrahman4u/seq2seq#144

A workaround for this is to only save the models weight. Then, when you want to load the model, you first recreate it with the same setup and then load the weights. See for details the link above.

[liehe]

@saritanavuluru
I met the same problem today : ** TypeError: init() missing 1 required positional arguments**. Here is how I solve the problem : (Keras 2.0.2)

1. Give the positional arguments of the layer with some default values
2. Override get_config function to the layer with some thing like

def get_config(self):
    config = super().get_config()
    config['localization_net'] = # say self. _localization_net  if you store the argument in __init__
    config['output_size'] = # say self. _output_size  if you store the argument in __init__
    return config

3. add layer class to custom_objects when you are loading model.

[Abhijit-2592]

@LiamHe I have the same issue. Am new to Keras. What do you mean by overriding get_config do you mind explaining it a bit more detailed?

[liehe]

@Abhijit-2592 Sorry that I can't elaborate the function for the moment. The function I wrote has bugs in some corner case. I will try to solve this problem over the weekend.

[alexrame]

@LiamHe
So I find a way to reload my spatial transformer layer using some tweaks
You need to define the deepcopy method for keras Sequential so that you do not end in an infinite loop: then the get_config will work
setattr(Sequential, '__deepcopy__', lambda self, _: self)

And you also need to modify the build method in SpatialTransformer so that you correctly build the Sequential Layer that defines your localizer

class SpatialTransformer(Layer):
def init(self,
localization_net=None,
output_size=None,
**kwargs):
self.locnet = localization_net
self.output_size = output_size
super(SpatialTransformer, self).init(**kwargs)

def build_locnet(self, input_shape):
    if isinstance(self.locnet, dict):
        self.locnet = Sequential.from_config([self.locnet])
    self.locnet.build(input_shape)

def build(self, input_shape):
    self.build_locnet(input_shape)
    self.trainable_weights = self.locnet.trainable_weights

def get_config(self):
    config = super(SpatialTransformer, self).get_config()
    config['localization_net'] = self.locnet
    # say self. _localization_net  if you store the argument in __init__
    config['output_size'] = self.output_size
    # say self. _output_size  if you store the argument in __init__

    return config

[Thrianadh]

I done in this way
from keras.models import Sequential
from keras_contrib.losses import import crf_loss
from keras_contrib.metrics import crf_viterbi_accuracy

To save model

model.save('my_model_01.hdf5')

To load the model

custom_objects={'CRF': CRF,'crf_loss':crf_loss,'crf_viterbi_accuracy':crf_viterbi_accuracy}

To load a persisted model that uses the CRF layer

model1 = load_model("/home/abc/my_model_01.hdf5", custom_objects = custom_objects)

[zhoudaxia233]

Use custom_objects to pass a dictionary to load_model. It isn't documented under load_model but it's documented under layer_from_config. This way you can load custom layers.

https://github.com/fchollet/keras/blob/master/keras/utils/layer_utils.py

Make sure to implement get_config() in your custom layer, it is used to save the model correctly.

If you have a lot of issues with load_model, save_weights and load_weights can be more reliable.

Cheers,
Ben

@bstriner Hello Ben, I'd like to know that, do I still need to pass custom_objects to load_model if I've already implemented get_config function in my custom layer? If so, what's the point here? My model also works without implementing get_config as long as I pass custom_objects.

Thanks a lot!

[shinleylee]

@zhoudaxia233 To my understanding, custom_objects is used to map your customized layer when load the model, while get_config is used to map and config the parameters within your layer. As for your model "also works without implementing get_config", I guess your layer didn't ask for any new arguments in __init__()? So it used the default get_config from its father class Layer.

[Walid-Ahmed]

This repo shows a simple sample code to build your own keras layer and use it in your model
https://github.com/Walid-Ahmed/kerasExamples/tree/master/creatingCustoumizedLayer
it might help

[0weights]

try to construct the model and then load the weights