Custom layers

[lenlen]

It seems that custom layers are not recognized in the train phase of spark model, specifically when loaded from yaml (model_from_yaml). I have tried with a custom activation function and it didn't work, the get_from_module function raised an exception "Invalid activation function: ...".

[maxpumperla]

Thanks. Is that maybe a problem specific to Keras, i.e. can you .to_yaml and model_from_yaml() outside of Elephas? Does that work for your layer with plain Keras?

[maxpumperla]

I'm asking because the get_from_module from Keras will probably only look into the activations submodule of core Keras.

[lenlen]

Thanks @maxpumperla, you are right, the model_from_yaml with custom activation function doesn't work in keras too.
This is an example code:

#model is the keras module
#my_function is my activation function
y_string = model.to_yaml()
new_model = model_from_yaml(y_string, custom_objects={"my_function":my_function})

Maybe this is not the right way to get the model with custom object... but I can't find an example of usage in keras documentation.

[maxpumperla]

Sure, no problem. If you really want to test this right now, fork Keras and put your special stuff there, i.e. in your case put the new activation next to the other keras activations. Then install your fork locally (cd your_keras && python setup.py install). This way there won't be a difference between custom and core.

Currently I see no quick way around this. You have to "register" your custom things somewhere, otherwise functionality like model_from_yaml simply has no way of knowing.

[maxpumperla]

Feel free to close, if you like

[lenlen]

Ok thanks, I closed this issue.

[lenlen]

Hi @maxpumperla I reopen this ticket because I have some news. Using a custom layer instead of custom activation function, the read from yaml works in keras but it doesn't work in elephas. I think that the problem is in spark_model.py when the model_from_yaml(..) is called during the train because it needs the custom_objects parameter. Using custom_objects parameter directly in keras works for example:

model = model_from_yaml(self.yaml,  custom_objects={"MyLayer":MyLayer, "myLoss":myLoss})

[deli4iled]

I've the same problem. I created a custom layer but it doesn't work.

[maxpumperla]

Alright, I will have to have a closer look at this. I'll have to see how to get the custom_objects parameter properly into an elephas model. Thanks for this, I'll get back to you

[FelipeMarcelino]

I have the same problem with custom loss function. Somone have a quick fix for this??

[danielenricocahall]

I will look into this - I think it will just be a matter of ensuring custom_objects can be optionally supplied every place we call it.

[danielenricocahall]

This is officially resolved in 1.2.0 - you'll just have to ensure to supply the custom_objects anywhere the model is loaded i.e;

    # define custom activation function 
    def custom_activation(x):
        ....
    # define model
     model = Sequential()
     model.add(Dense(1, input_dim=1, activation=custom_activation))
     ...
    model.add(Dense(1, activation='sigmoid'))
    spark_model = SparkModel(model, frequency='epoch', mode=mode,
                             custom_objects={'custom_activation': custom_activation})

Same process applies for a custom layer.In the context of an estimator:

    # define custom activation function 
    def custom_activation(x):
        ....
    # define model
     model = Sequential()
     model.add(Dense(1, input_dim=1, activation=custom_activation))

    estimator = ElephasEstimator()
    estimator.set_keras_model_config(model.to_yaml())
    estimator.set_optimizer_config(sgd_conf)
    ...
    estimator.set_epochs(10)
    estimator.set_batch_size(32)
    estimator.set_validation_split(0.01)
    estimator.set_custom_objects({'custom_activation': custom_activation}) # provide custom_objects in setter method