Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add caffe scale #732

Merged
merged 1 commit into from Sep 11, 2019

Conversation

@rainLiuplus
Copy link
Collaborator

commented Sep 11, 2019

No description provided.

Rain

@rainLiuplus rainLiuplus merged commit b5feba9 into master Sep 11, 2019

2 checks passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details
license/cla All CLA requirements met.
Details

@rainLiuplus rainLiuplus deleted the add-Affine branch Sep 11, 2019

@bersbersbers

This comment has been minimized.

Copy link

commented Sep 12, 2019

While you are at this: I believe a change in mapper.py like this would be useful, too:

- kwargs = {'scale' : len(node.data) >= 3, 'bias' : len(node.data) == 4}
+ kwargs = {'scale' : len(node.data) >= 3, 'bias' : len(node.data) == 4} if node.data else {'scale' : False, 'bias' : False}

This prevents errors in case node.data is None.

@bersbersbers

This comment has been minimized.

Copy link

commented Sep 12, 2019

Also, why does the Scale layer test for node.data at all, if it is not using node.data anywhere? Shouldn't it be testing for node.parameters, if anything? (This may be a stupid questions, but my example have empty node.data for some reason.)
And finally, why does the Scale (which expected one bottom) layer fall back to Mul (which expects two bottoms)?

@bersbersbers

This comment has been minimized.

Copy link

commented Sep 12, 2019

When I load a model created using this code, I receive an error in generic_utils.py", line 140, deserialize_keras_object:

ValueError: Unknown layer: Affine

I am not aware of an Affine layer - how can I use this model? I went about to define Affine as in the commit, then issue

import keras
keras.layers.Affine=Affine

But I had to change it to def __init__(self, scale=1, bias=0, **kwargs):, as None was not supported in the model.

@bersbersbers

This comment has been minimized.

Copy link

commented Sep 12, 2019

Finally, the Affine layer is not marked as having an parameters in a Keras model summary. This is unintended, I would guess.

@rainLiuplus

This comment has been minimized.

Copy link
Collaborator Author

commented Sep 12, 2019

  1. the caffe [Scale] can have more than one bottom: https://github.com/BVLC/caffe/blob/master/src/caffe/layers/scale_layer.cpp#L128-L129
    and has its scale param: https://github.com/BVLC/caffe/blob/master/src/caffe/layers/scale_layer.cpp#L14

    Therefore, the judgment of the existence of param is omitted here, and the node.data actually is the scale layer blob, thus the reason why test it is shown in the first link.

  2. you could make a PR to perfect the keras [Affine] or implement any idea you think is better.

@bersbersbers

This comment has been minimized.

Copy link

commented Sep 12, 2019

  1. Thanks for the answers! I didn't know about the flexibility of the Scale layer.

  2. Regarding improving the Affine layer, I am currently using the conversion to test my understanding of Caffe batch normalization/scaling, and just need some code to convert - so I won't be able to improve it myself.
    But I do see the issue that the Keras models cannot be loaded with load_model due to the None issue, and the layer cannot be trained either. Should I open separate issues about this?

  3. Finally, my apparent issue with node.data being empty occurs only in combination with a batch normalization layer. Here's one example that fails, should I open separate issues about this?

pip3 install -U git+https://github.com/Microsoft/MMdnn.git@b5feba92e0f539a9e070315e2211a5311a56efbb
mmconvert -sf caffe -in Bug.prototxt -df keras -om Temp

It outputs:

I0912 13:58:26.298133   383 layer_factory.hpp:77] Creating layer input
I0912 13:58:26.298238   383 net.cpp:84] Creating Layer input
I0912 13:58:26.298338   383 net.cpp:380] input -> data
I0912 13:58:26.298467   383 net.cpp:122] Setting up input
I0912 13:58:26.298554   383 net.cpp:129] Top shape: 1 3 10 10 (300)
I0912 13:58:26.298650   383 net.cpp:137] Memory required for data: 1200
I0912 13:58:26.298728   383 layer_factory.hpp:77] Creating layer norm
I0912 13:58:26.298816   383 net.cpp:84] Creating Layer norm
I0912 13:58:26.298897   383 net.cpp:406] norm <- data
I0912 13:58:26.298979   383 net.cpp:380] norm -> datanorm
I0912 13:58:26.299105   383 net.cpp:122] Setting up norm
I0912 13:58:26.299201   383 net.cpp:129] Top shape: 1 3 10 10 (300)
I0912 13:58:26.299299   383 net.cpp:137] Memory required for data: 2400
I0912 13:58:26.299428   383 layer_factory.hpp:77] Creating layer scale
I0912 13:58:26.299537   383 net.cpp:84] Creating Layer scale
I0912 13:58:26.299633   383 net.cpp:406] scale <- datanorm
I0912 13:58:26.299731   383 net.cpp:380] scale -> datanormscale
I0912 13:58:26.299865   383 layer_factory.hpp:77] Creating layer scale
I0912 13:58:26.299994   383 net.cpp:122] Setting up scale
I0912 13:58:26.300091   383 net.cpp:129] Top shape: 1 3 10 10 (300)
I0912 13:58:26.300187   383 net.cpp:137] Memory required for data: 3600
I0912 13:58:26.300288   383 net.cpp:200] scale does not need backward computation.
I0912 13:58:26.300393   383 net.cpp:200] norm does not need backward computation.
I0912 13:58:26.300490   383 net.cpp:200] input does not need backward computation.
I0912 13:58:26.300587   383 net.cpp:242] This network produces output datanormscale
I0912 13:58:26.300686   383 net.cpp:255] Network initialization done.
Type                 Name                                          Param               Output
----------------------------------------------------------------------------------------------
Data                 data                                             --       (1, 3, 10, 10)
BatchNorm            norm                                             --       (1, 3, 10, 10)
Scale                scale                                            --       (1, 3, 10, 10)
Traceback (most recent call last):
  File "/home/bers/.local/bin/mmconvert", line 11, in <module>
    load_entry_point('mmdnn==0.2.4', 'console_scripts', 'mmconvert')()
  File "/home/bers/.local/lib/python3.6/site-packages/mmdnn/conversion/_script/convert.py", line 102, in _main
    ret = convertToIR._convert(ir_args)
  File "/home/bers/.local/lib/python3.6/site-packages/mmdnn/conversion/_script/convertToIR.py", line 17, in _convert
    graph = transformer.transform_graph()
  File "/home/bers/.local/lib/python3.6/site-packages/mmdnn/conversion/caffe/transformer.py", line 367, in transform_graph
    mapped_node = self.map_node(node)
  File "/home/bers/.local/lib/python3.6/site-packages/mmdnn/conversion/caffe/transformer.py", line 389, in map_node
    mapped_node = map_func(node)
  File "/home/bers/.local/lib/python3.6/site-packages/mmdnn/conversion/caffe/mapper.py", line 251, in map_batch_norm
    kwargs = {'scale' : len(node.data) >= 3, 'bias' : len(node.data) == 4}
TypeError: object of type 'NoneType' has no len()

Bug.prototxt:

input: "data"
input_shape {
  dim: 1
  dim: 3
  dim: 10
  dim: 10
}
layer {
  bottom: 'data'
  top: 'datanorm'
  name: 'norm'
  type: 'BatchNorm'
  batch_norm_param {
    use_global_stats: false
    moving_average_fraction: 0.999
    eps: 1e-5
  }
  param {
    lr_mult: 0 
  }
  param {
    lr_mult: 0
  }
  param {
    lr_mult: 0
  }
}
layer {
  bottom: 'datanorm'
  top: 'datanormscale'
  name: 'scale'
  type: 'Scale'
  scale_param {
    axis: 1
    filler: {
      type: 'constant' 
      value: 1
    }
    bias_term: true
    bias_filler {
      type: 'constant'
      value: 0.
    }
  }
}
@rainLiuplus

This comment has been minimized.

Copy link
Collaborator Author

commented Sep 12, 2019

--> 2. , thanks for your reminder. I think I should modify the implementation.

-->3. , I think the reason is that you did not write the weight path (-iw xxx.caffemodel), so the node.data defaults to None.

Finally, please feel free to open new issues.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.