[Keras] Support MXNET backend for Keras. #4173

shivarajugowda · 2016-12-09T17:09:34Z

I have been working on supporting MXNET as a backend for Keras(A popular neural networks Python library which currently supports TensorFlow or Theano). I am hopeful the endeavor is win-win for both projects. Keras will benefit from the MXNET’s multi-device/multi-node support and MXNET will get broader exposure. The task will also test out and most probably enhance MXNET API capabilities for a broader set of audience.

To this end, I have started the process and am able to get low-hanging APIs checked off. I could say I have 25% of the work done. The amount of changes required in Keras is not huge we just need to add one more file on the same lines as of tensorflow_backend.py
I think most of the work would be in figuring out how to map the functionalities to MXNET APIs and implementing the missing ones.

The backend tests are a good way to go about implementing it and tracking our progress.

Here is my rough current status as measured in terms of those tests.

APIs converted:

test_linear_operations
test_shape_operations
test_elementwise_operations
test_nn_operations
test_value_manipulation
test_random_normal
test_random_uniform
test_random_binomial

APIs I am currently working on.

test_gradient
test_rnn
test_rnn_no_states
test_conv2d
test_conv3d
test_pool2d
test_pool3d

APIs I think might need changes/updates to MXNET.

test_function
test_ctc
test_ctc_decode_greedy
test_ctc_decode_beam_search
test_one_hot
test_sparse_dot
test_sparse_concat
test_map
test_foldl
test_foldr

Things that are currently missing but are nice to have:

Python distutil distribution for MXNET.
Provide core functionality in NDArray. For some of the functionalities I had to use Symbol module.

First things first, I want to know if this is inline with the MXNET community’s needs and something which you agree needs to be pursued and is worthwhile. If we agree then I can use the issue as a high-level task to track and update my progress and also request more info/features and help with implementing them.

piiswrong · 2016-12-09T19:16:31Z

@shivarajugowda
Thanks for the wonderful effort!
Yes this is something we want. In fact we have been talking about doing this for a while but didn't have the man power.

Could you post your code somewhere so we can track progress and other people interested in this can pitch in?

piiswrong · 2016-12-09T19:27:54Z

Also please work with the nnvm branch (instead of master) since it will be released soon.

with nnvm branch all functions are available in both symbol and ndarray.

shivarajugowda · 2016-12-09T20:33:22Z

@piiswrong Good to know this would be of value. Yes I will start checking in (whatever I have), but all the changes as of now are in Keras branch. I will point to it in here once I checkin(by Monday). As and when I need changes to the MXNET I will create a separate issue and summarize it here. Also appreciate pointing out to use NNVM branch, I was using the master branch till now. I will check out the new symbol in NNVM branch.

ivenzor · 2016-12-10T08:42:59Z

Good work!

anjishnu · 2016-12-10T22:21:12Z

+1, great work. Would love to see this happen.

jspisak · 2016-12-12T04:20:50Z

This is really great! Thanks @shivarajugowda for jumping in on this..

shivarajugowda · 2016-12-12T05:59:34Z

Here is where you can monitor the progress on the Keras end.

Branch : https://github.com/shivarajugowda/keras
Issue: keras-team/keras#1313

shivarajugowda · 2016-12-12T06:15:05Z

Adding support of element wise !=, >, >=, <, <=comparison operators as a part of this. #4182

shivarajugowda · 2016-12-12T20:06:09Z

For Keras, we need support for Sparse Matrices, I see a proposal in #1524. Not sure how far off we are in terms of progress. For the time being I am using the Dense matrix underneath.

shivarajugowda · 2016-12-15T16:44:29Z

Filed #4248 and #4249 to support mean and std deviation operators.

fchollet · 2016-12-16T22:34:10Z

For Keras, we need support for Sparse Matrices

If you leave out sparse tensor support, very little functionality would be lost. As long as you raise appropriate, helpful exceptions in the backend, it would be fine.

Do you foresee any issues with K.rnn, K.gradients? These two would be the tricky ones.

shivarajugowda · 2016-12-16T23:40:19Z

@fchollet thanks for the input, I am figuring out loop and if constructs in the context of RNN. Apart from MXNET, I am also looking at Tensorflow and Theano code. Will get more time during the Christmas break, will keep the updates posted here.

piiswrong · 2016-12-17T06:31:28Z

gradient is in there (gradient pass) but not exposed through c api. @tqchen @jermainewang Is it possible to expose it?

tqchen · 2016-12-19T16:54:02Z

There are essentially two approaches possible. Keras takes a pure declarative symbolic approach for both network definition and parameter update, because the existing frameworks on keras works are declarative.

However, it is not necessarily to be so in MXNet, and the parameter update part is automatically handled with imperative code. They do not necessarily be incompatible with keras API, most of keras API are for the network definition generations.

So I would suggest the follow approach

Make network configuration API fully compatible with keras
Wrap the update logic with the imperative module API(they are not long anyway).

I know this may take a bit of additional effort, but it also takes benefit from mutli GPU API available in mxnet module.

As a second approach, we can reuse the gradient pass in mxnet and take a purely declarative approach, which I view will take a bit more effort, and may not directly come with multi-gpu support.

tqchen · 2016-12-19T16:58:34Z

My comment do not be block any of the existing issues, but instead break the goals into two parts (and two layers of compatibility)

1. Compatibility of network graph generation API(without requirement of gradient, and scan)
- The model fitting logic swapped by module API in mxnet, with API compatibility
1. Compatibility of the underlying execution code, gradient generation and optimization
  - I feel this part could be more framework dependent, especially when it comes to multiGPU and multi machine

I am all for both directions, but break it into two part will make the milestone easier, I am in favor of quickly achieving 1, so everything is functioning, and possibly stabbing 2 later.

shivarajugowda · 2016-12-20T01:48:47Z

@tqchen I am probably not familiar with the terminology used in the MXNET context. I could follow some broader thought but I couldn't follow all of the details. @tqchen, @piiswrong How about a Webex/Hangout to go over this and also validate that I am in the right direction. Let me know and I can setup one if and when you have some time.

piiswrong · 2016-12-20T07:47:05Z

@shivarajugowda Yes we should have a meeting. What time zone are you in?

shivarajugowda · 2016-12-20T16:29:18Z

@piiswrong I am in Pacific Time Zone(CA, bay area). I am available anytime tomorrow.

piiswrong · 2016-12-20T19:47:49Z

my email is eric.jy.xie@gmail.com

…

On Tue, Dec 20, 2016 at 8:29 AM, Shiv Gowda ***@***.***> wrote: @piiswrong <https://github.com/piiswrong> I am in Pacific Time Zone(CA, bay area). I am available anytime tomorrow. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#4173 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAiudJkak1plQmybnCocMGQPLS2jzR9Iks5rKAJtgaJpZM4LJJ3K> .

shivarajugowda · 2016-12-23T19:42:17Z

I have integrated conv2d and pool2d.
MXNET is missing support for 3D Convolution (#4301) and Pooling with 3D kernel.

shivarajugowda · 2016-12-23T20:49:36Z

NDArray.onehot_encode() only support indices in 1D. We need support for multiple dimension.

shivarajugowda · 2017-01-05T17:08:51Z

Update:
I have mapped a few more operators(2D convolution/pooling, map_fn, foldl, foldr, etc) and I am pursuing @tqchen and @piiswrong suggestion of "Compatibility of network graph generation API(without requirement of gradient, and scan) The model fitting logic swapped by module API in mxnet, with API compatibility" for a simple example of "Keras/keras/examples/mnist_mlp.py". The operators are mapped for this example and I am working on using the MXNET module api underneath the Keras Model.fit().

imranshaikmuma · 2017-07-20T23:11:45Z

is this still open? what is the progress? does keras has api for mxnet now?
i dont see in keras documentation. I like MXNET context thing!!! please let me if it is available on keras through mxnet backend

shivarajugowda · 2017-07-20T23:20:10Z

This issue can be closed now, the dmlc folks have a fork of Keras working with MXNET. https://github.com/dmlc/keras
https://medium.com/@julsimon/apache-mxnet-support-in-keras-83de7dec46e5

imranshaikmuma · 2017-07-20T23:55:03Z

i am getting the following error:

shivarajugowda changed the title ~~Support MXNET backend for Keras.~~ [Keras] Support MXNET backend for Keras. Dec 11, 2016

piiswrong added the focus progress label Dec 12, 2016

shivarajugowda mentioned this issue Dec 12, 2016

mxnet as backend keras-team/keras#1313

Closed

shivarajugowda mentioned this issue Dec 12, 2016

Sparse Matrix Specification #1524

Closed

shivarajugowda closed this as completed Jul 20, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Keras] Support MXNET backend for Keras. #4173

[Keras] Support MXNET backend for Keras. #4173

shivarajugowda commented Dec 9, 2016

piiswrong commented Dec 9, 2016 •

edited by mli

Loading

piiswrong commented Dec 9, 2016 •

edited

Loading

shivarajugowda commented Dec 9, 2016

ivenzor commented Dec 10, 2016

anjishnu commented Dec 10, 2016

jspisak commented Dec 12, 2016

shivarajugowda commented Dec 12, 2016

shivarajugowda commented Dec 12, 2016

shivarajugowda commented Dec 12, 2016

shivarajugowda commented Dec 15, 2016

fchollet commented Dec 16, 2016

shivarajugowda commented Dec 16, 2016 •

edited

Loading

piiswrong commented Dec 17, 2016 •

edited

Loading

tqchen commented Dec 19, 2016

tqchen commented Dec 19, 2016

shivarajugowda commented Dec 20, 2016

piiswrong commented Dec 20, 2016

shivarajugowda commented Dec 20, 2016

piiswrong commented Dec 20, 2016 via email

shivarajugowda commented Dec 23, 2016 •

edited

Loading

shivarajugowda commented Dec 23, 2016

shivarajugowda commented Jan 5, 2017

imranshaikmuma commented Jul 20, 2017 •

edited

Loading

shivarajugowda commented Jul 20, 2017

imranshaikmuma commented Jul 20, 2017

[Keras] Support MXNET backend for Keras. #4173

[Keras] Support MXNET backend for Keras. #4173

Comments

shivarajugowda commented Dec 9, 2016

piiswrong commented Dec 9, 2016 • edited by mli Loading

piiswrong commented Dec 9, 2016 • edited Loading

shivarajugowda commented Dec 9, 2016

ivenzor commented Dec 10, 2016

anjishnu commented Dec 10, 2016

jspisak commented Dec 12, 2016

shivarajugowda commented Dec 12, 2016

shivarajugowda commented Dec 12, 2016

shivarajugowda commented Dec 12, 2016

shivarajugowda commented Dec 15, 2016

fchollet commented Dec 16, 2016

shivarajugowda commented Dec 16, 2016 • edited Loading

piiswrong commented Dec 17, 2016 • edited Loading

tqchen commented Dec 19, 2016

tqchen commented Dec 19, 2016

shivarajugowda commented Dec 20, 2016

piiswrong commented Dec 20, 2016

shivarajugowda commented Dec 20, 2016

piiswrong commented Dec 20, 2016 via email

shivarajugowda commented Dec 23, 2016 • edited Loading

shivarajugowda commented Dec 23, 2016

shivarajugowda commented Jan 5, 2017

imranshaikmuma commented Jul 20, 2017 • edited Loading

shivarajugowda commented Jul 20, 2017

imranshaikmuma commented Jul 20, 2017

piiswrong commented Dec 9, 2016 •

edited by mli

Loading

piiswrong commented Dec 9, 2016 •

edited

Loading

shivarajugowda commented Dec 16, 2016 •

edited

Loading

piiswrong commented Dec 17, 2016 •

edited

Loading

shivarajugowda commented Dec 23, 2016 •

edited

Loading

imranshaikmuma commented Jul 20, 2017 •

edited

Loading