Lambda layers in the keras model

[gnsmrky]

Continued test from issue #41, it seems like the exported keras model has lambda op. Is it correct?

output of print (k_model.summary()) in #41

Layer (type)                 Output Shape              Param #   
=================================================================
input0 (InputLayer)          (None, 3, 224, 224)       0         
_________________________________________________________________
lambda_1 (Lambda)            (None, 3, 232, 232)       0         <-- lambda layer
_________________________________________________________________
conv1.conv2d0.19608361958321 (None, 32, 224, 224)      7808      
_________________________________________________________________
in10.5890283736035935 (Lambd (None, 32, 224, 224)      0         <-- lambda layer
_________________________________________________________________
relu0.8399978862622054 (Acti (None, 32, 224, 224)      0         
=================================================================
Total params: 7,808
Trainable params: 7,808
Non-trainable params: 0

Trying to make the generated keras model file to tfjs (which does not support lambda layer). Knowing that lambda can be any python codes. This would make it not portable to non-python frameworks, such as c++ or js based ones.

Don't know if it's logical to ask, but is it possible to workaround this requirements for lambda layer?

[gmalivenko]

hi @gnsmrky. Yes, Keras don't have InstanceNorm2d layers so, it was converted as a lambdas. If you want to convert it to tfjs, you can simply get tensorflow session and convert it directly from tensorflow (as a frozen graph).

[gmalivenko]

k_model = pytorch2keras(....)
k_model.summary()  # see the last layer name
from keras import backend as K
import tensorflow as tf
frozen_graph = freeze_session(K.get_session(),
                              output_names=[out.op.name for out in k_model.outputs])

tf.train.write_graph(frozen_graph, ".", "my_model.pb", as_text=False)

You know input layers names, you know output layers names, so, you can call tfjs converter:

For example:

tensorflowjs_converter \
     --input_format=tf_frozen_model \
     --output_node_names='FCesQtnC/BiasAdd' \       
     my_model.pb \
     my_model

[gnsmrky]

Ah yes, I noticed that it seems to using lambda to convert unsupported keras layers.

I actually also tried additional tensorflow exports, such as tf.train.Saver.save() and tf.train.write_graph(). Yet it went into another problem in tfjs converter:
ValueError: Unsupported Ops in the model before optimization MirrorPad

Is it the correct behavior that MirrorPad is generated?

[gmalivenko]

Are you sure that MirrorPad is really supported by tfjs?

 self.reflection_pad = torch.nn.ReflectionPad2d(reflection_padding)

I think there is no another way to convert this layer except MirrorPad.

[gnsmrky]

tfjs converter does not support MirrorPad (it does not support InstanceNorm2d, either). The question should be better asked that why ReflectionPad2d gets translated to MirrorPad, as tf.pad should already support REFLECT mode. (See tf.pad doc.)

Took a quick look at pytorch2keras layers.py. It does seem like the 'REFLECT' mode is already being handled by tf.pad at pytorch2keras/layers.py#L1292. Strange that it gets converted into MirrorPad layer.

pytorch2keras/layers.py#L1292:
layer = tf.pad(x, [[0, 0], [0, 0], [pads[2], pads[6]], [pads[3], pads[7]]], 'REFLECT').

The only difference here from the tf.pad doc is that it has the additional [0,0],[0,0] in the paddings param. Would it be the reason why the MirrorPad is generated instead of Pad layer?

[gmalivenko]

I afraid it's tensorflow issue: MirrorPad.

[gnsmrky]

@nerox8664 Thanks for the finding. I guess the file format does not reflect what's at the top level. Should've realized that earlier.

Next logical step, I guess, is to see what can be tweaked at the graph to avoid MirrorPad.