New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add CNTK as keras backend #6800

Merged
merged 101 commits into from Jun 7, 2017

Conversation

Projects
None yet
10 participants
@souptc
Contributor

souptc commented May 30, 2017

This is the Beta version of CNTK as keras backend. Those changes are suppose to work with CNTK v2.0 GA. Because the CNTKv2.0 is not officially published, you could use the wheel below to have a first try:

http://cntk.ai/PythonWheel/ForKeras/cntk-2.0rc3-cp35-cp35m-linux_x86_64.whl
http://cntk.ai/PythonWheel/ForKeras/cntk-2.0rc3-cp27-cp27mu-linux_x86_64.whl

Most of the keras features are supported, except:

  1. Performance optimization on CPU device.
    CNTK performance optimization on cpu device is not finished, so if you want better performance, please try to run CNTK_Keras with GPU device. 
     
  2. Gradient as symbolic ops.
    This is not include in CNTK_Keras beta release, and we will support it in CNTK_Keras GA release. 
     
  3. Stateful recurrent layer.
    This is not include in CNTK_Keras beta release. 
     
  4. Masking on recurrent layer.
    This is not include in CNTK_Keras beta release, and we will support it in CNTK_Keras GA release. 
     
  5. Padding with non-specified shape. 
    Padding with non-speicfied shape is not supported now. To using cntk backend in keras with padding, please specified the concrete input shape. 
     
  6. Convolution with dilation.
    This is not include in CNTK_Keras beta release, and we will support it in CNTK_Keras GA release. 
     
  7. Randomness op across batch axis. 
    This is not include in CNTK_Keras beta release, and we will support it in CNTK_Keras GA release. 
     
  8. Some Keras backend apis: reverse / top_k / ctc / map / foldl / foldr. 
    These are not include in CNTK_Keras beta release, and we will support it in CNTK_Keras GA release. 
@souptc

This comment has been minimized.

Show comment
Hide comment
@souptc

souptc Jun 3, 2017

Contributor

@fchollet , I just update the implementation which resolve the most comments. For the "bias_shape", I explained the purpose of the change, do you have any suggestion about a better solution?

Contributor

souptc commented Jun 3, 2017

@fchollet , I just update the implementation which resolve the most comments. For the "bias_shape", I explained the purpose of the change, do you have any suggestion about a better solution?

@souptc

This comment has been minimized.

Show comment
Hide comment
@souptc

souptc Jun 3, 2017

Contributor

And for the maintainance, yes, CNTK team will definitely commit to maintain the cntk backend in the future.

Contributor

souptc commented Jun 3, 2017

And for the maintainance, yes, CNTK team will definitely commit to maintain the cntk backend in the future.

chentaMS and others added some commits Jun 5, 2017

Add sparse_top_k_categorical_accuracy and test code (#6840)
* Add top_k_sparse_categorical_accuracy and test_top_k_sparse_categorical_accuracy

* Rename top_k_sparse_categorical_accuracy and sparse_top_k_categorical_accuracy
@fchollet

The file cntk_backend will require some cleanups and a style normalization. E.g. it mixes different quote characters for string delimitation, and other small style issues.

Show outdated Hide outdated LICENSE
Show outdated Hide outdated keras/backend/tensorflow_backend.py
@souptc

This comment has been minimized.

Show comment
Hide comment
@souptc

souptc Jun 6, 2017

Contributor

clean the cntk_backend file with style issue.

Contributor

souptc commented Jun 6, 2017

clean the cntk_backend file with style issue.

@souptc

This comment has been minimized.

Show comment
Hide comment
@souptc

souptc Jun 6, 2017

Contributor

fix the bias_shape issue.

Contributor

souptc commented Jun 6, 2017

fix the bias_shape issue.

@fchollet

Some style nitpicks. Almost ready to merge!

Show outdated Hide outdated keras/backend/cntk_backend.py
Show outdated Hide outdated keras/backend/tensorflow_backend.py
Show outdated Hide outdated keras/backend/tensorflow_backend.py
Show outdated Hide outdated keras/backend/tensorflow_backend.py
Show outdated Hide outdated keras/backend/tensorflow_backend.py
Show outdated Hide outdated keras/backend/theano_backend.py
Show outdated Hide outdated keras/backend/theano_backend.py
Show outdated Hide outdated keras/backend/theano_backend.py
Show outdated Hide outdated keras/backend/theano_backend.py
Show outdated Hide outdated keras/layers/local.py
@fchollet

This comment has been minimized.

Show comment
Hide comment
@fchollet

fchollet Jun 7, 2017

Collaborator

The failing test is a flake.

Collaborator

fchollet commented Jun 7, 2017

The failing test is a flake.

@fchollet fchollet merged commit 82832a3 into keras-team:master Jun 7, 2017

1 check failed

continuous-integration/travis-ci/pr The Travis CI build failed
Details
@fchollet

This comment has been minimized.

Show comment
Hide comment
@fchollet

fchollet Jun 7, 2017

Collaborator

I have applied a few style fixes to the backend source and merged. Thanks a lot! 👍

While going through the backend code, I have noticed that a lot of the error messages raised were not as helpful and as clear as they should be. I suggest you go over the error messages and improve them. Typically you want to tell the user: 1) what they did (e.g. print the arguments they passed), 2) why that was wrong (e.g. something not supported), and 3) what they should do instead. Currently you are mostly doing 2) only.

This will greatly improve the user experience for CNTK Keras users, and improve the usability / ease of debugging of Keras models on CNTK.

Collaborator

fchollet commented Jun 7, 2017

I have applied a few style fixes to the backend source and merged. Thanks a lot! 👍

While going through the backend code, I have noticed that a lot of the error messages raised were not as helpful and as clear as they should be. I suggest you go over the error messages and improve them. Typically you want to tell the user: 1) what they did (e.g. print the arguments they passed), 2) why that was wrong (e.g. something not supported), and 3) what they should do instead. Currently you are mostly doing 2) only.

This will greatly improve the user experience for CNTK Keras users, and improve the usability / ease of debugging of Keras models on CNTK.

@souptc

This comment has been minimized.

Show comment
Hide comment
@souptc

souptc Jun 7, 2017

Contributor

Thanks François! Sure, I will go though the message tomorrow and improve them.

Contributor

souptc commented Jun 7, 2017

Thanks François! Sure, I will go though the message tomorrow and improve them.

@ebarsoumMS

This comment has been minimized.

Show comment
Hide comment
@ebarsoumMS

ebarsoumMS Jun 7, 2017

Thanks, everybody...

ebarsoumMS commented Jun 7, 2017

Thanks, everybody...

@fchollet

This comment has been minimized.

Show comment
Hide comment
@fchollet

fchollet Jun 7, 2017

Collaborator

Huge thanks to everyone who contributed to this project! It's a big milestone.

Collaborator

fchollet commented Jun 7, 2017

Huge thanks to everyone who contributed to this project! It's a big milestone.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment