Add documentation for loading keras models #2005
Conversation
| @@ -0,0 +1,24 @@ | |||
| # **Keras Support** | |||
There was a problem hiding this comment.
It can be moved to https://bigdl-project.github.io/master/#APIGuide/Module/
There was a problem hiding this comment.
keep it here just like tf support/caffe support. That issue is not related with this PR.
|
@jason-dai any comments on the doc? |
| keras_model.add(Activation('relu')) | ||
| keras_model.add(Dropout(0.5)) | ||
| keras_model.add(Dense(10)) | ||
| keras_model.add(Activation('softmax')) |
There was a problem hiding this comment.
We should use LogSoftMax together with ClassNLL, or we can refactor this later after this PR (#1944 )merged
| def_path = "lenet.json" | ||
| with open(def_path, "w") as json_file: | ||
| json_file.write(model_json) | ||
|
|
There was a problem hiding this comment.
Divide this into 3 : 1) Pure keras model definition + data processing 2) save keras model 3)load keras model and train with BigDL optimizer. Maybe extract 3 methods to make your intention more clear.
| model=bigdl_model, | ||
| training_rdd=train_data, | ||
| criterion=ClassNLLCriterion(), | ||
| optim_method=SGD(learningrate=0.01), |
docs/docs/APIGuide/Module.md
Outdated
|
|
||
| __Remark__: Packages `tensorflow`, `keras==1.2.2` and `h5py` are required. They can be installed via `pip` easily. | ||
|
|
||
| ```python |
docs/docs/APIGuide/Module.md
Outdated
|
|
||
| You can directly use the API `load_keras` to load the Keras model into BigDL. | ||
|
|
||
| __Remark__: Packages `tensorflow`, `keras==1.2.2` and `h5py` are required. They can be installed via `pip` easily. |
There was a problem hiding this comment.
why tensorflow is required?
There was a problem hiding this comment.
tensorflow is the backend used by keras
There was a problem hiding this comment.
is one of the backend. For keras model save/load, I don't think we need tensorflow
|
|
||
| You can directly use the API `load_keras` to load the Keras model into BigDL. | ||
|
|
||
| __Remark__: Packages `tensorflow`, `keras==1.2.2` and `h5py` are required. They can be installed via `pip` easily. |
|
|
||
| __Remark__: Packages `tensorflow`, `keras==1.2.2` and `h5py` are required. They can be installed via `pip` easily. | ||
|
|
||
| ```python |
There was a problem hiding this comment.
where is the scala example?
| ### **Limitations** | ||
| We have tested the model loading functionality with several standard [Keras applications](https://faroit.github.io/keras-docs/1.2.2/applications/) and [examples](https://github.com/fchollet/keras/tree/1.2.2/examples). | ||
|
|
||
| However, there still exist some arguments for Keras layers that are not supported in BigDL for now. Also we haven't supported self-defined Keras layers, but one can still define your customized layer converter and weight converter method for new layers if you wish. |
There was a problem hiding this comment.
Can we create a page provide the list of keras layer we have supported? like this: https://github.com/tf-coreml/tf-coreml#supported-ops
|
jenkins passed http://172.168.2.101:8080/job/BigDL-PR-V/532/ |
|
Due to the limitation of our implementation, loading a keras model into BigDL need to rely on keras (mainly used for the calculation of input shape). In other words, if users need to use this functionality, they need to install |
|
Since keras needs a backend to construct the model, in the future we may consider using bigdl backend (only implement the minimum necessary methods) as suggested by Jason. |
|
LGTM |
1 similar comment
|
LGTM |
…s#2005) * update yarn doc * update according to comments
What changes were proposed in this pull request?
Add doc for keras model loading.