Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

load_weights will fail if shape mismatch #10266

Merged
merged 1 commit into from May 24, 2018
Merged

Conversation

@AmirAlavi
Copy link
Contributor

AmirAlavi commented May 22, 2018

Fix for #10265
Ran keras tests, same results as origin/master:

31 failed, 549 passed, 98 skipped, 162 warnings, 2 error

Fix for #10265
@taehoonlee

This comment has been minimized.

Copy link
Member

taehoonlee commented May 23, 2018

It is reasonable, but I am concerned about performance degradation. If shape checking is performed for every weight, load_weights will be very slow.

@AmirAlavi

This comment has been minimized.

Copy link
Contributor Author

AmirAlavi commented May 23, 2018

Fair point, @taehoonlee. I timed it with and without my changes to load weights for ResNet50 with this gist: https://gist.github.com/AmirAlavi/ab8e3bdf3fca09c25ba808dd4cbdd48a

Time without my changes: 0.013439978134818375
Time with my changes: 0.013317499115131795

This was with timeit, with 10 repetitions of 100 calls (reported time is the minimum from the 10 runs, divided by 100). It's odd that it was faster (please double check my timing method), but I think the times are so similar that we can take this to mean that the performance hit is negligible if present at all. Please let me know what you think.

@AmirAlavi

This comment has been minimized.

Copy link
Contributor Author

AmirAlavi commented May 23, 2018

@taehoonlee, sorry, I had a mistake in my timing script (skip_mismatch was set to True, which caused the original code to also check each shape) which explains why the times were so similar. I had set it to True to get a particular model of mine to work, which had shape mismatches. I just changed it to skip_mismatch=False and here are the results:

Time without my changes: 0.013299283728003502
Time with my changes: 0.01338828501291573

It's slightly slower, but I think it's worth pointing out that in the current code, the different values of skip_mismatch cause different runtimes of load_weights in a non-intuitive way: if I "skip" things, I expect them to be faster, but load_weights is actually (slightly) slower because of the shape checking.

With this pull request, the runtimes will be the same regardless of the value of that flag, and we gain proper reporting of programming error.

Copy link
Collaborator

fchollet left a comment

LGTM

@fchollet fchollet merged commit 5eecd55 into keras-team:master May 24, 2018
1 check passed
1 check passed
continuous-integration/travis-ci/pr The Travis CI build passed
Details
sandeep-krishnamurthy added a commit to awslabs/keras-apache-mxnet that referenced this pull request Jun 15, 2018
davidtvs added a commit to davidtvs/keras that referenced this pull request Jun 19, 2018
commit 65670e6
Author: David Silva <davidtvs10@gmail.com>
Date:   Tue Jun 19 06:00:55 2018 +0100

    Replace two spaces by br tag to force new line

commit d714193
Author: David Silva <davidtvs10@gmail.com>
Date:   Tue Jun 19 05:26:57 2018 +0100

    Add output shape equation to docstrings

commit 5f06a78
Author: David Silva <davidtvs10@gmail.com>
Date:   Sat Jun 16 17:26:20 2018 +0100

    Fixes following @fchollet revision

commit b4530ae
Merge: 6b99d23 5fcd832
Author: David Silva <davidtvs10@gmail.com>
Date:   Sat Jun 16 16:17:41 2018 +0100

    Merge remote-tracking branch 'upstream/master' into trans_out_pad_dev

commit 5fcd832
Author: Pavithra Vijay <psv@google.com>
Date:   Fri Jun 15 16:29:32 2018 -0700

    Fix small doc error (keras-team#10453)

commit 2ec486b
Author: Francois Chollet <francois.chollet@gmail.com>
Date:   Fri Jun 15 14:07:28 2018 -0700

    Model input validation fix.

commit ffa99d6
Author: Pavithra Vijay <psv@google.com>
Date:   Fri Jun 15 13:52:14 2018 -0700

    Fix use_bias in some convolutional layers (keras-team#10444)

    * Fix use_bias in some convolutional layers

    * Fix Conv2DTranspose test

commit 6aace30
Author: ebatuhankaynak <ebatuhankaynak@users.noreply.github.com>
Date:   Fri Jun 15 22:00:24 2018 +0300

    Update convolutional.py (keras-team#10452)

    Changed
    "# now model.output_shape == (None, 20, 16. 64)"
    to
    "# now model.output_shape == (None, 20, 16, 64)"

    (There is a dot instead of comma).

commit 2d183db
Author: Sachin Abeywardana <sachinruk@users.noreply.github.com>
Date:   Fri Jun 15 04:19:45 2018 +1000

    corrected batch norm implementation in cntk (keras-team#10427)

commit 52f608c
Author: Taehoon Lee <me@taehoonlee.com>
Date:   Thu Jun 14 05:44:22 2018 +0900

    Add tests for inputs set dynamically (keras-team#10416)

commit a68c516
Author: Taehoon Lee <me@taehoonlee.com>
Date:   Thu Jun 14 03:43:43 2018 +0900

    Update a CNTK version on Travis (keras-team#10419)

commit 4f90f95
Author: ebatuhankaynak <ebatuhankaynak@users.noreply.github.com>
Date:   Wed Jun 13 21:39:47 2018 +0300

    Update faq.md (keras-team#10422)

    Changed "Notes that the methods...." to "Note that the methods"

commit 8e5b853
Author: Taehoon Lee <me@taehoonlee.com>
Date:   Thu Jun 14 03:39:13 2018 +0900

    Move `get_source_inputs` (keras-team#10415)

commit a40f335
Author: Max Pumperla <max.pumperla@googlemail.com>
Date:   Wed Jun 13 20:36:50 2018 +0200

    [docs] more details for adagrad/delta, clarifying usage of rho (keras-team#10410)

    * some more meat for ada docs

    * incorporate suggestions

    * line breaks

commit 29a22a8
Author: fuzzythecat <fuzzy0427@gmail.com>
Date:   Wed Jun 13 02:57:38 2018 +0900

    Add missing named arguments in ImageDataGenerator in examples (keras-team#10389)

commit 0237542
Author: EyeBool <diataha.personal@gmail.com>
Date:   Mon Jun 11 09:44:02 2018 -0700

    Fixed a type: repeated 'in common'. (keras-team#10394)

commit 3eab610
Author: Francois Chollet <francois.chollet@gmail.com>
Date:   Thu Jun 7 16:09:58 2018 -0700

    Preserve input shape data when serializing deferred-build Sequential models.

commit db849f8
Author: jlopezpena <jlopezpena@users.noreply.github.com>
Date:   Thu Jun 7 19:54:51 2018 +0100

    Doc update (keras-team#10376)

    A couple of variables are "used" in two examples without being defined. For consistency with other examples where auxiliary dimensions are defined, I think it would be better to explicitly assign them a value. I just used made up values, feel free to change to whatever makes more sense!

commit 67e242d
Author: Max Pumperla <max.pumperla@googlemail.com>
Date:   Thu Jun 7 20:54:23 2018 +0200

    Add spatial dropout and 3D global pooling to docs (keras-team#10373)

    * spatial dropout in docs

    * 3d global pooling in docs

commit 4e82971
Author: Francois Chollet <francois.chollet@gmail.com>
Date:   Wed Jun 6 16:17:29 2018 -0700

    Update docs for 2.2.0.

commit 632560d
Author: Francois Chollet <francois.chollet@gmail.com>
Date:   Wed Jun 6 11:53:18 2018 -0700

    Style fix

commit bbf4283
Author: Bohumír Zámečník <bohumir.zamecnik@gmail.com>
Date:   Wed Jun 6 20:52:26 2018 +0200

    CuDNN RNN layers nested in TimeDistributed are not converted when loading (keras-team#10357)

    * Add a unit test for CuDNNGRU conversion with TimeDistributed.

    * Extract duplicated function convert_model() to _convert_model_weights().

    * keras-team#10356 Convert weights of CuDNN/plain RNN nested in TimeDistributed.

    Same case as for Bidirectional, except that in TimeDistributed there's only
    one nested layer instead of two.

commit 13548e8
Author: Taehoon Lee <me@taehoonlee.com>
Date:   Thu Jun 7 03:51:37 2018 +0900

    Add tests for inputs set dynamically (keras-team#10367)

commit 56b255c
Author: Francois Chollet <francois.chollet@gmail.com>
Date:   Wed Jun 6 10:51:06 2018 -0700

    Update to Keras Applications 1.0.2 (fixes NASNet issue).

commit 513f7b1
Author: Taehoon Lee <me@taehoonlee.com>
Date:   Thu Jun 7 00:47:37 2018 +0900

    Fix a version number (keras-team#10361)

commit 9a58f7b
Author: Francois Chollet <francois.chollet@gmail.com>
Date:   Tue Jun 5 17:40:44 2018 -0700

    Prepare 2.2.0 release.

commit 3f4ccbb
Author: Francois Chollet <francois.chollet@gmail.com>
Date:   Tue Jun 5 17:30:48 2018 -0700

    Add error message when calling `summary` on unbuilt subclassed models.

commit 25d0193
Author: tiferet <tgazit@gmail.com>
Date:   Tue Jun 5 21:23:27 2018 +0300

    Supporting channels_first data format with crossentropy losses (keras-team#9715)

commit 1e80c1a
Author: KuzMenachem <35711942+KuzMenachem@users.noreply.github.com>
Date:   Tue Jun 5 20:10:43 2018 +0200

     Fix HDF5Matrix issue when working in conjunction with TimeSeriesGenerator (keras-team#10334)

    * Fix issue when working in conjunction with TimeSeriesGenerator

    The TimeSeriesGenerator class uses xrange through six which caused an IndexError

    * Add test

    * Add corresponding test

    * Fix for python3

    * Simplified code

    * Fix indent

    * Fix test

commit ce56322
Author: David Schwertfeger <david.schwertfeger@googlemail.com>
Date:   Tue Jun 5 00:15:25 2018 +0200

    FIX: Tensorboard callback only supports logging Embeddings layer weights (keras-team#7766)

    * Embed layer-outputs rather than layer-weights in TensorBoard callback

    * Update docstring and allow multiple inputs

    * Fix tests

    * Renaming

    * Set learning phase

    * Compute embeddings in batches

    * Pass embedding data explicitly

    * Actually process embeddings in batches

    * Allow multiple inputs and validate input data

    * Add example

    * Delete utils.py

    * Revert uncorrectly resolved merge-conflict

    * Minor renaming

    * Add comment clarifying the design choice

commit b217648
Author: Tommi Koivisto <tkoivisto@users.noreply.github.com>
Date:   Mon Jun 4 20:26:41 2018 +0300

    Add an advanced activation layer for ReLU (keras-team#10322)

    The max_value argument can not be used in a layer, except
    custom layer or Lambda. Hence, similarly to LeakyReLU or
    for example Softmax, this PR adds a layer for ReLU,
    enabling also a capped ReLU to be used.

commit 1365ed5
Author: Taehoon Lee <me@taehoonlee.com>
Date:   Tue Jun 5 02:07:23 2018 +0900

    Reduce tests for applications (keras-team#10346)

    * Reduce tests for applications

    * Make selection over all models random

commit 52e3f98
Author: Anirudh Swaminathan <aniswami97@gmail.com>
Date:   Sun Jun 3 01:22:09 2018 +0800

    Improve the docstring of Conv3DTranspose (keras-team#10342)

    * Add depth as third dimension in docstring of
      Conv3DTranspose in convolutional.py in keras.layers

commit f89b339
Author: Taehoon Lee <me@taehoonlee.com>
Date:   Sun Jun 3 02:21:06 2018 +0900

    Move tests for applications (keras-team#10341)

commit c9cb608
Author: François Chollet <francois.chollet@gmail.com>
Date:   Fri Jun 1 23:53:49 2018 -0700

    Split `applications` and `preprocessing` modules. (keras-team#10339)

    * Split `applications` and `preprocessing` modules.

    * Fix dependencies.

commit a366424
Author: Zhengping Che <Peterche1990@Hotmail.com>
Date:   Sat Jun 2 07:13:24 2018 +0800

    Handle `mask` in `TimeDistributed` wrapper. (keras-team#10242)

    * equip TimeDistributed with mask and unspecified input length

    * fix bugs in theano. add test on timedistributed + masking

    * skip tests on cntk with multiple unspecified time lengths.

    * move static shape inference to theano_backend, add docstring, etc.

    * fix format

commit 7365a99
Author: fuzzythecat <fuzzy0427@gmail.com>
Date:   Fri Jun 1 04:45:26 2018 +0900

    Fix doc (keras-team#10327)

    Fixed doc

commit fe06696
Author: fuzzythecat <fuzzy0427@gmail.com>
Date:   Wed May 30 13:49:44 2018 +0900

    Add pooling options in MobileNetV2 (keras-team#10313)

    * Add pooling option

    * Add pooling test

commit 315a80a
Author: Taehoon Lee <me@taehoonlee.com>
Date:   Wed May 30 01:43:37 2018 +0900

    Improve docstrings of applications (keras-team#10310)

commit 25283ea
Author: Francois Chollet <francois.chollet@gmail.com>
Date:   Tue May 29 09:31:14 2018 -0700

    Fix typo

commit 0de2adf
Author: fuzzythecat <fuzzy0427@gmail.com>
Date:   Tue May 29 13:58:18 2018 +0900

    Fix doc  (keras-team#10308)

    * Fix naming convention

    * Add missing doc

commit 9bff5b1
Author: r-kellerm <34206121+r-kellerm@users.noreply.github.com>
Date:   Mon May 28 03:45:35 2018 +0300

    Typo fix (keras-team#10293)

commit 794f814
Author: Botty Dimanov <bottydim@users.noreply.github.com>
Date:   Fri May 25 22:21:41 2018 +0100

    Handle capitalised extensions in list_pictures (keras-team#10220)

    keras-team#10219

commit 84aa7b5
Author: Wang, Zhiming <zhiming.wang@intel.com>
Date:   Sat May 26 05:20:57 2018 +0800

    Non training Batch Norm operator has bad performance for it running into tensorflow's non fused batch norm API (keras-team#10207)

    * When use tensorflow as backend, let batch norm run into fused batch norm as much as possible, which has better performance.

    fix issue: http://github.com/keras-team/keras/issues/10058

    * In Tensorflow backend, let batch norm call to FusedBatchNorm only NHWC format, also gamma and beta are not None.

    Test result:
    test env: with Tensorflow(commit a543d9471047ca3f6881c87105fcbe2cdff9207d Date:   Thu May 10 17:43:30 2018, local build), python3.4, centos7.4
    test cases:
      "pytest  ./tests/keras/layers/normalization_test.py"  <all passed>
      "pytest  ./tests"      <keep same result as without this commit's modification on BN>

    * fix code sytle.

    * 1. Add axis parameter in backend's batch_normalization functions.
    2. Refine the batch_normalization function in tensorflow backend, Let's it call to fused batch norm as much as possible.

    Thanks the coments from fchollet.

    * Trigger

    * 1. add default value -1 for parameter axis in batch_normalization function in backend.
    2. fix some code style.
    Thanks the comments from fchollet.

commit e6d2179
Author: Stanley Bileschi <bileschi@google.com>
Date:   Fri May 25 17:19:38 2018 -0400

    Adds to and alphabetizes documentation of Layer base class. (keras-team#10282)

    * Alphabetizes and adds to layers doc.

    * Responding to @Cais comments

    * fix spacing.  Remove in(out)bound_nodes

commit 5eecd55
Author: Amir Alavi <s.amir.alavi@gmail.com>
Date:   Thu May 24 16:54:20 2018 -0400

    load_weights will fail if shape mismatch (keras-team#10266)

    Fix for keras-team#10265

commit d476ecc
Author: jnphilipp <mail@jnphilipp.org>
Date:   Thu May 24 19:52:11 2018 +0200

    Use count_params function for non_trainable_count. (keras-team#10280)

commit a07d9f3
Author: Stefano Cappellini <contact@stefanocappellini.com>
Date:   Thu May 24 19:50:21 2018 +0200

    Fixes automatic doc generation problem with nested lists. Adds a new test (keras-team#10212)

    * Fixes automatic doc generation problem with indented lists. Adds a new test

    * Some style fixes on doc automatic generation files

    * Fixes a bad space in convolutional_recurrent.py

    * Changes the test_doc_auto_generation in order to include a doc string taken from the codebase. Allows text lines following nested lists

commit 5a48df2
Author: Clemens Tolboom <clemens@build2be.com>
Date:   Thu May 24 00:00:42 2018 +0200

    Make Dot documentation inline with Concatenate (keras-team#10271)

    Doc expects a list containing 2 tensors.

commit 2064477
Author: askskro <ak@aiascience.com>
Date:   Wed May 23 23:59:58 2018 +0200

    Typo in docstring for softplus (keras-team#10277)

    Softplus docstring missing a parenthesis.

commit c77267a
Author: Brian Nemsick <brian.nemsick@gmail.com>
Date:   Wed May 23 11:54:36 2018 -0700

    Remove deprecated model.model from engine/saving (keras-team#10275)

commit 08c8736
Author: vkk800 <vkk800@users.noreply.github.com>
Date:   Tue May 22 23:03:51 2018 +0200

    Refactor ImageDataGenerator (keras-team#10130)

    * Create get_random_transform and refactor

    * Fix style and add tests

    * Add more tests

    * Fix documentation error

    * Fix documentation style issue

    * add apply_affine_transform

    * document transformation dictionary

    * Doc style fix

commit 25a8973
Author: Taehoon Lee <me@taehoonlee.com>
Date:   Wed May 23 04:18:46 2018 +0900

    Increase test coverages by factorizing CNTK pads (keras-team#10259)

commit abd0294
Author: brge17 <33430930+brge17@users.noreply.github.com>
Date:   Mon May 21 18:42:07 2018 -0700

    Remove Sequential.model deprecation warning (keras-team#10256)

    * Remove Sequential.model deprecation warning

    * Remove dead line of code

commit 2eed2a5
Author: Taehoon Lee <me@taehoonlee.com>
Date:   Tue May 22 03:11:43 2018 +0900

    Increase test coverages by adding invalid CNTK usecases (keras-team#10236)
fchollet added a commit that referenced this pull request Jun 21, 2018
* Add output shape argument to transposed conv. to resolve ambiguity

* Remove deconv2d  legacy preprocessor

* Add unit tests for output shape in 2d/3d transpose

* Replace output_shape with output_padding

* Update docstrings

* deconv_length handles output_padding=None

* Unit tests

* Check if stride is greater than output padding

* Pull request revision

commit 65670e6
Author: David Silva <davidtvs10@gmail.com>
Date:   Tue Jun 19 06:00:55 2018 +0100

    Replace two spaces by br tag to force new line

commit d714193
Author: David Silva <davidtvs10@gmail.com>
Date:   Tue Jun 19 05:26:57 2018 +0100

    Add output shape equation to docstrings

commit 5f06a78
Author: David Silva <davidtvs10@gmail.com>
Date:   Sat Jun 16 17:26:20 2018 +0100

    Fixes following @fchollet revision

commit b4530ae
Merge: 6b99d23 5fcd832
Author: David Silva <davidtvs10@gmail.com>
Date:   Sat Jun 16 16:17:41 2018 +0100

    Merge remote-tracking branch 'upstream/master' into trans_out_pad_dev

commit 5fcd832
Author: Pavithra Vijay <psv@google.com>
Date:   Fri Jun 15 16:29:32 2018 -0700

    Fix small doc error (#10453)

commit 2ec486b
Author: Francois Chollet <francois.chollet@gmail.com>
Date:   Fri Jun 15 14:07:28 2018 -0700

    Model input validation fix.

commit ffa99d6
Author: Pavithra Vijay <psv@google.com>
Date:   Fri Jun 15 13:52:14 2018 -0700

    Fix use_bias in some convolutional layers (#10444)

    * Fix use_bias in some convolutional layers

    * Fix Conv2DTranspose test

commit 6aace30
Author: ebatuhankaynak <ebatuhankaynak@users.noreply.github.com>
Date:   Fri Jun 15 22:00:24 2018 +0300

    Update convolutional.py (#10452)

    Changed
    "# now model.output_shape == (None, 20, 16. 64)"
    to
    "# now model.output_shape == (None, 20, 16, 64)"

    (There is a dot instead of comma).

commit 2d183db
Author: Sachin Abeywardana <sachinruk@users.noreply.github.com>
Date:   Fri Jun 15 04:19:45 2018 +1000

    corrected batch norm implementation in cntk (#10427)

commit 52f608c
Author: Taehoon Lee <me@taehoonlee.com>
Date:   Thu Jun 14 05:44:22 2018 +0900

    Add tests for inputs set dynamically (#10416)

commit a68c516
Author: Taehoon Lee <me@taehoonlee.com>
Date:   Thu Jun 14 03:43:43 2018 +0900

    Update a CNTK version on Travis (#10419)

commit 4f90f95
Author: ebatuhankaynak <ebatuhankaynak@users.noreply.github.com>
Date:   Wed Jun 13 21:39:47 2018 +0300

    Update faq.md (#10422)

    Changed "Notes that the methods...." to "Note that the methods"

commit 8e5b853
Author: Taehoon Lee <me@taehoonlee.com>
Date:   Thu Jun 14 03:39:13 2018 +0900

    Move `get_source_inputs` (#10415)

commit a40f335
Author: Max Pumperla <max.pumperla@googlemail.com>
Date:   Wed Jun 13 20:36:50 2018 +0200

    [docs] more details for adagrad/delta, clarifying usage of rho (#10410)

    * some more meat for ada docs

    * incorporate suggestions

    * line breaks

commit 29a22a8
Author: fuzzythecat <fuzzy0427@gmail.com>
Date:   Wed Jun 13 02:57:38 2018 +0900

    Add missing named arguments in ImageDataGenerator in examples (#10389)

commit 0237542
Author: EyeBool <diataha.personal@gmail.com>
Date:   Mon Jun 11 09:44:02 2018 -0700

    Fixed a type: repeated 'in common'. (#10394)

commit 3eab610
Author: Francois Chollet <francois.chollet@gmail.com>
Date:   Thu Jun 7 16:09:58 2018 -0700

    Preserve input shape data when serializing deferred-build Sequential models.

commit db849f8
Author: jlopezpena <jlopezpena@users.noreply.github.com>
Date:   Thu Jun 7 19:54:51 2018 +0100

    Doc update (#10376)

    A couple of variables are "used" in two examples without being defined. For consistency with other examples where auxiliary dimensions are defined, I think it would be better to explicitly assign them a value. I just used made up values, feel free to change to whatever makes more sense!

commit 67e242d
Author: Max Pumperla <max.pumperla@googlemail.com>
Date:   Thu Jun 7 20:54:23 2018 +0200

    Add spatial dropout and 3D global pooling to docs (#10373)

    * spatial dropout in docs

    * 3d global pooling in docs

commit 4e82971
Author: Francois Chollet <francois.chollet@gmail.com>
Date:   Wed Jun 6 16:17:29 2018 -0700

    Update docs for 2.2.0.

commit 632560d
Author: Francois Chollet <francois.chollet@gmail.com>
Date:   Wed Jun 6 11:53:18 2018 -0700

    Style fix

commit bbf4283
Author: Bohumír Zámečník <bohumir.zamecnik@gmail.com>
Date:   Wed Jun 6 20:52:26 2018 +0200

    CuDNN RNN layers nested in TimeDistributed are not converted when loading (#10357)

    * Add a unit test for CuDNNGRU conversion with TimeDistributed.

    * Extract duplicated function convert_model() to _convert_model_weights().

    * #10356 Convert weights of CuDNN/plain RNN nested in TimeDistributed.

    Same case as for Bidirectional, except that in TimeDistributed there's only
    one nested layer instead of two.

commit 13548e8
Author: Taehoon Lee <me@taehoonlee.com>
Date:   Thu Jun 7 03:51:37 2018 +0900

    Add tests for inputs set dynamically (#10367)

commit 56b255c
Author: Francois Chollet <francois.chollet@gmail.com>
Date:   Wed Jun 6 10:51:06 2018 -0700

    Update to Keras Applications 1.0.2 (fixes NASNet issue).

commit 513f7b1
Author: Taehoon Lee <me@taehoonlee.com>
Date:   Thu Jun 7 00:47:37 2018 +0900

    Fix a version number (#10361)

commit 9a58f7b
Author: Francois Chollet <francois.chollet@gmail.com>
Date:   Tue Jun 5 17:40:44 2018 -0700

    Prepare 2.2.0 release.

commit 3f4ccbb
Author: Francois Chollet <francois.chollet@gmail.com>
Date:   Tue Jun 5 17:30:48 2018 -0700

    Add error message when calling `summary` on unbuilt subclassed models.

commit 25d0193
Author: tiferet <tgazit@gmail.com>
Date:   Tue Jun 5 21:23:27 2018 +0300

    Supporting channels_first data format with crossentropy losses (#9715)

commit 1e80c1a
Author: KuzMenachem <35711942+KuzMenachem@users.noreply.github.com>
Date:   Tue Jun 5 20:10:43 2018 +0200

     Fix HDF5Matrix issue when working in conjunction with TimeSeriesGenerator (#10334)

    * Fix issue when working in conjunction with TimeSeriesGenerator

    The TimeSeriesGenerator class uses xrange through six which caused an IndexError

    * Add test

    * Add corresponding test

    * Fix for python3

    * Simplified code

    * Fix indent

    * Fix test

commit ce56322
Author: David Schwertfeger <david.schwertfeger@googlemail.com>
Date:   Tue Jun 5 00:15:25 2018 +0200

    FIX: Tensorboard callback only supports logging Embeddings layer weights (#7766)

    * Embed layer-outputs rather than layer-weights in TensorBoard callback

    * Update docstring and allow multiple inputs

    * Fix tests

    * Renaming

    * Set learning phase

    * Compute embeddings in batches

    * Pass embedding data explicitly

    * Actually process embeddings in batches

    * Allow multiple inputs and validate input data

    * Add example

    * Delete utils.py

    * Revert uncorrectly resolved merge-conflict

    * Minor renaming

    * Add comment clarifying the design choice

commit b217648
Author: Tommi Koivisto <tkoivisto@users.noreply.github.com>
Date:   Mon Jun 4 20:26:41 2018 +0300

    Add an advanced activation layer for ReLU (#10322)

    The max_value argument can not be used in a layer, except
    custom layer or Lambda. Hence, similarly to LeakyReLU or
    for example Softmax, this PR adds a layer for ReLU,
    enabling also a capped ReLU to be used.

commit 1365ed5
Author: Taehoon Lee <me@taehoonlee.com>
Date:   Tue Jun 5 02:07:23 2018 +0900

    Reduce tests for applications (#10346)

    * Reduce tests for applications

    * Make selection over all models random

commit 52e3f98
Author: Anirudh Swaminathan <aniswami97@gmail.com>
Date:   Sun Jun 3 01:22:09 2018 +0800

    Improve the docstring of Conv3DTranspose (#10342)

    * Add depth as third dimension in docstring of
      Conv3DTranspose in convolutional.py in keras.layers

commit f89b339
Author: Taehoon Lee <me@taehoonlee.com>
Date:   Sun Jun 3 02:21:06 2018 +0900

    Move tests for applications (#10341)

commit c9cb608
Author: François Chollet <francois.chollet@gmail.com>
Date:   Fri Jun 1 23:53:49 2018 -0700

    Split `applications` and `preprocessing` modules. (#10339)

    * Split `applications` and `preprocessing` modules.

    * Fix dependencies.

commit a366424
Author: Zhengping Che <Peterche1990@Hotmail.com>
Date:   Sat Jun 2 07:13:24 2018 +0800

    Handle `mask` in `TimeDistributed` wrapper. (#10242)

    * equip TimeDistributed with mask and unspecified input length

    * fix bugs in theano. add test on timedistributed + masking

    * skip tests on cntk with multiple unspecified time lengths.

    * move static shape inference to theano_backend, add docstring, etc.

    * fix format

commit 7365a99
Author: fuzzythecat <fuzzy0427@gmail.com>
Date:   Fri Jun 1 04:45:26 2018 +0900

    Fix doc (#10327)

    Fixed doc

commit fe06696
Author: fuzzythecat <fuzzy0427@gmail.com>
Date:   Wed May 30 13:49:44 2018 +0900

    Add pooling options in MobileNetV2 (#10313)

    * Add pooling option

    * Add pooling test

commit 315a80a
Author: Taehoon Lee <me@taehoonlee.com>
Date:   Wed May 30 01:43:37 2018 +0900

    Improve docstrings of applications (#10310)

commit 25283ea
Author: Francois Chollet <francois.chollet@gmail.com>
Date:   Tue May 29 09:31:14 2018 -0700

    Fix typo

commit 0de2adf
Author: fuzzythecat <fuzzy0427@gmail.com>
Date:   Tue May 29 13:58:18 2018 +0900

    Fix doc  (#10308)

    * Fix naming convention

    * Add missing doc

commit 9bff5b1
Author: r-kellerm <34206121+r-kellerm@users.noreply.github.com>
Date:   Mon May 28 03:45:35 2018 +0300

    Typo fix (#10293)

commit 794f814
Author: Botty Dimanov <bottydim@users.noreply.github.com>
Date:   Fri May 25 22:21:41 2018 +0100

    Handle capitalised extensions in list_pictures (#10220)

    #10219

commit 84aa7b5
Author: Wang, Zhiming <zhiming.wang@intel.com>
Date:   Sat May 26 05:20:57 2018 +0800

    Non training Batch Norm operator has bad performance for it running into tensorflow's non fused batch norm API (#10207)

    * When use tensorflow as backend, let batch norm run into fused batch norm as much as possible, which has better performance.

    fix issue: http://github.com/keras-team/keras/issues/10058

    * In Tensorflow backend, let batch norm call to FusedBatchNorm only NHWC format, also gamma and beta are not None.

    Test result:
    test env: with Tensorflow(commit a543d9471047ca3f6881c87105fcbe2cdff9207d Date:   Thu May 10 17:43:30 2018, local build), python3.4, centos7.4
    test cases:
      "pytest  ./tests/keras/layers/normalization_test.py"  <all passed>
      "pytest  ./tests"      <keep same result as without this commit's modification on BN>

    * fix code sytle.

    * 1. Add axis parameter in backend's batch_normalization functions.
    2. Refine the batch_normalization function in tensorflow backend, Let's it call to fused batch norm as much as possible.

    Thanks the coments from fchollet.

    * Trigger

    * 1. add default value -1 for parameter axis in batch_normalization function in backend.
    2. fix some code style.
    Thanks the comments from fchollet.

commit e6d2179
Author: Stanley Bileschi <bileschi@google.com>
Date:   Fri May 25 17:19:38 2018 -0400

    Adds to and alphabetizes documentation of Layer base class. (#10282)

    * Alphabetizes and adds to layers doc.

    * Responding to @Cais comments

    * fix spacing.  Remove in(out)bound_nodes

commit 5eecd55
Author: Amir Alavi <s.amir.alavi@gmail.com>
Date:   Thu May 24 16:54:20 2018 -0400

    load_weights will fail if shape mismatch (#10266)

    Fix for #10265

commit d476ecc
Author: jnphilipp <mail@jnphilipp.org>
Date:   Thu May 24 19:52:11 2018 +0200

    Use count_params function for non_trainable_count. (#10280)

commit a07d9f3
Author: Stefano Cappellini <contact@stefanocappellini.com>
Date:   Thu May 24 19:50:21 2018 +0200

    Fixes automatic doc generation problem with nested lists. Adds a new test (#10212)

    * Fixes automatic doc generation problem with indented lists. Adds a new test

    * Some style fixes on doc automatic generation files

    * Fixes a bad space in convolutional_recurrent.py

    * Changes the test_doc_auto_generation in order to include a doc string taken from the codebase. Allows text lines following nested lists

commit 5a48df2
Author: Clemens Tolboom <clemens@build2be.com>
Date:   Thu May 24 00:00:42 2018 +0200

    Make Dot documentation inline with Concatenate (#10271)

    Doc expects a list containing 2 tensors.

commit 2064477
Author: askskro <ak@aiascience.com>
Date:   Wed May 23 23:59:58 2018 +0200

    Typo in docstring for softplus (#10277)

    Softplus docstring missing a parenthesis.

commit c77267a
Author: Brian Nemsick <brian.nemsick@gmail.com>
Date:   Wed May 23 11:54:36 2018 -0700

    Remove deprecated model.model from engine/saving (#10275)

commit 08c8736
Author: vkk800 <vkk800@users.noreply.github.com>
Date:   Tue May 22 23:03:51 2018 +0200

    Refactor ImageDataGenerator (#10130)

    * Create get_random_transform and refactor

    * Fix style and add tests

    * Add more tests

    * Fix documentation error

    * Fix documentation style issue

    * add apply_affine_transform

    * document transformation dictionary

    * Doc style fix

commit 25a8973
Author: Taehoon Lee <me@taehoonlee.com>
Date:   Wed May 23 04:18:46 2018 +0900

    Increase test coverages by factorizing CNTK pads (#10259)

commit abd0294
Author: brge17 <33430930+brge17@users.noreply.github.com>
Date:   Mon May 21 18:42:07 2018 -0700

    Remove Sequential.model deprecation warning (#10256)

    * Remove Sequential.model deprecation warning

    * Remove dead line of code

commit 2eed2a5
Author: Taehoon Lee <me@taehoonlee.com>
Date:   Tue May 22 03:11:43 2018 +0900

    Increase test coverages by adding invalid CNTK usecases (#10236)

* Revert "Pull request revision"

This reverts commit fb7c367.

* Pull request revision (v2)

* Replace HTML markup by markdown
sandeep-krishnamurthy added a commit to awslabs/keras-apache-mxnet that referenced this pull request Jun 21, 2018
* Document that "same" is inconsistent across backends with strides!=1 (#9629)

* Document that `"same"` is inconsistent across backends with `strides` != 1

* Use "[here](...)"

* Add h5py to dependencies

* import `pydot`, improve error messages about `pydot` and GraphViz, bump to `pydot >= 1.2.4` (#9904)

* REL: bump to `pydot >= 1.2.4` in `extras_require`

* MAI: import pydot (as required in `extras_require`)

* MAI: refine error messages for `pydot` and GraphViz

distinguish between absence of `pydot` and failure to find
the executables of GraphViz in the $PATH.

* DEV: ignore `.pytest_cache`

* Added note to manually install h5py where needed (#9830)

* Added notes to manually install h5py if needed

* Added FAQ entry on h5py

* deleted redundant remark about h5py

* updated FAQ to reflect dependency change

* fixed comment format to pass failing test

* removed new trailing whitespaces

* improved docstring format

* reverted callbacks.py

* fixed links in model.py

* updated faq.py

* link pointing to FAQ

* Added an error message for undefined shape on NASNet. (#9891)

* Added an error message for undefined shape on NASNet.

* Forgot that the message should be present only when loading imagenet weights.

* Changed the message.

* Fix PEP8

* Make conv_invalid_use and pooling_invalid_use efficient (#9944)

* Chenta/cntk bn (#9952)

* fix cntk static learning phase issue; add a test

* fix code style;add more comments

* add boolean support

* fix code style issue

* fixing typos (#10016)

* Add `separable_conv2d` for Theano (#10003)

* Add `separable_conv2d` for Theano

* Enable the layer test for `separable_conv2d`

* Fix the strides for 1x1 conv

* Refactor topological part of `engine` module (#10023)

* Refactor topological part of Keras engine.

* Fix imports

* Fix merge mixup.

* Fix Bidirectional Regularization (#10012)

* Fix Bidirectional Regularization

Override the Wrapper `get_updates_for` and `get_losses_for` methods so that contributions from both the forward and backward layers are included in Bidirectional.

* Use Parent for Calculating Inner Inputs

Remove unneeded inner input calculations.

* Simplify Bidirectional Losses

* Add Bidirectional Unit Tests

Test Bidirectional updates and losses.

* Remove Trailing Whitespace

* Fix Bidirectional Loss Inputs

* Add Tests for Conditional Updates/Losses

* Remove Whitespace

* Refactor training part of `engine` module. (#10029)

* Refactor topological part of Keras engine.

* Fix imports

* Fix merge mixup.

* Refactor training part of the Keras engine.

* Fix unit tests.

* Fix `int_shape` of Theano and Refactor associated lines (#10030)

* Add exceptions for `batch_dot` (#10020)

* Enable Xception to work on Theano and CNTK (#10024)

* Enable Xception to work on Theano and CNTK

* Fix different predictions over all the backends

* Add support for passthrough arguments to NumpyArrayIterator (#10035)

* Add support for second output to NumpyArrayIterator

* Enable Model subclassing API (#10046)

* Refactor topological part of Keras engine.

* Fix imports

* Fix merge mixup.

* Refactor training part of the Keras engine.

* Fix unit tests.

* Refactor Network to prepare support for Model subclassing

* Finish enabling Model subclassing.

* Add `long_description` field in setup.py.

* Remove unused import

* Increase pytest duration from 10 mins to 20 mins (#10072)

* [RELNOTES] Simplify implementation of Sequential and remove legacy Merge support (#10077)

* Refactor topological part of Keras engine.

* Fix imports

* Fix merge mixup.

* Refactor training part of the Keras engine.

* Fix unit tests.

* Refactor Network to prepare support for Model subclassing

* Finish enabling Model subclassing.

* Simplify Sequential implementation.

RELNOTES: This breaks weight loading and model loading/saving for models from Keras 0.* that used Merge layers. The Merge layer has been deprecated for 2 years.

* [RELNOTES] Remove support for Keras 0.* Merge layer and associated functionality, which was scheduled for 08/2017.

* fix typo (#10078)

* Add documentation to several activation functions (#10066)

* Add documentation to several activation functions

* Fix style issues.

* Small style fixes.

* [RELNOTES] Introduce `preprocessing.image.save_img` and remove deprecated imsave method in neural style transfer example (#9996)

* imsave method in scipy.misc package is deprecated - now using imageio

* updated save_img method to use array_to_img method.  also updated the neural style transfer example to use the new save_img method

* forgot to commit changes - updated save_img

* added file_format and **kwargs parameter to save_img and updated docstring

* removed space at the end of a line in save_img method.  updated instances of imsave in example scripts with save_img method.

* added kwargs to docstring.  removed additional whitespace lines

* removed trailing whitespace

* Extensive style fixes in `preprocessing.image`.

* [RELNOTES] Allow loading external backends (#10034)

* load external backend

* safety++

* Excpetion -> ValueError

* Revert TF version to 1.7 on Travis CI. (#10101)

* New Callback: EarlyBaselineStopping (#10061)

* Initial Commit

* Continued changes

* Alpha version

* Added support to make sure that previous epochs, which may pass the
baseline test, are kept in variable history.

* Code formatting

* Code formatting

* Code formatting to address CI errors

* Code formatting for CI errors

* Initial unit test code

* Adjust for epoch being zero-based

* Code formatting

* Unit tests added

* Code formatting

* Code formatting

* Factorized the unit test

* Code formatting

* Refactored to be part of EarlyStopping and modified unit tests

* Code formatting

* Adds MobiletNetV2 to applications (#10047)

* [RELNOTES] Allow symbolic tensors to be fed to models (with TF backend) (#10087)

* Allow symbolic tensors to be fed to models (with TF backend)

* Skip tensor feeding tests when TF>=1.8 not available.

* [RELNOTES] Fix EarlyStopping API

* Fix shape mismatch in `rnn()` of tensorflow_backend (#10038)

* Fix shape mismatch in `rnn()` of tensorflow_backend

Fix the shape mismatch in `rnn()` with `not unroll` and `mask is not None` of the Tensorflow backend.

Problem: If the rnn cell has any (recurrent) `states[i]` whose shape is different from that of the `output` (`states[0]`), there will raise an `ValueError` when updating that state.

Reason: This is because the `tiled_mask_t` is not updated correctly for each `states[i]` but instead is simply copied from that of the `output` (`states[0]`).

Solution: Tile the `mask_t` with the correct shape of each `states[i]`. Notice that in a similar situation with `unroll is True`, the `tiled_mask_t` is handled correctly.

* add unit_test for rnn() with states whose shape is different from that of the output.

* Revert "add unit_test for rnn() with states whose shape is different from that of the output."

This reverts commit f1df2a58ff635bbf698444e3d7403785a92dfed1.

* Simplify the unit_test for rnn with additional states

* [RELNOTES] Default `Flatten` layer’s `data_format` argument to `None`.

* #10080 Convert CuDNN weights in nested Model. (#10081)

* #10080 Convert CuDNN weights in nested Model.

- similar problem to nesting in Bidirectional (#8860)
- quick fix: just copy/paste. I'll refactor it later
- also convert biases from H5 Dataset to np.array for reshape() (#9662)

* Refactor keras.engine.saving.preprocess_weights_for_loading().

- less duplication
- extracted conversion methods for Bidirectional/Model/Sequential

* Format docstrings to follow the project code style.

* Move tests of CuDNN RNN weight conversion to a more proper place.

- move from cudnn_recurrent_test.py to test_model_saving.py
- adjust imports to be consistent

* Make better tests for CuDNN RNN weight conversion.

- add nested models (#10080)
- various model types: add Model, Sequential (not only Sequential)
- convert weights on model, not layer (not test nesting in models)
  - test just by save/load model weights instead of calling preprocess_weights_for_loading() directly

* Check GPU support via tf in pytest skipif with short-circuit evaluation.

It seems that multiple skipif decorators get evaluated eagerly, not lazily.
Thus it fails on theano and cntk backends.

I don't understand why the tests didn't fail until now.
Maybe some change in pytest?

* Refactor (deduplicate) skipif decorators for TensorFlow and GPU.

At least multiple occurrences within a test module.
Now we can't import from `tests/` and `keras.utils.test_utils` don't se pytest.
Otherwise we can define the marker only once at all.

* Fix PEP8 (whitespace).

* Add `separable_conv1d` (#10125)

* Add `separable_conv1d` for Theano

* Add `separable_conv1d` for CNTK

* Fix preprocess_input not working with int arrays (#10134)

* Make MobileNet tests random (#10132)

* Sample weighted ImageDataGenerator (#10092)

* Add support for sample_weight in ImageDataGenerator.flow

* Added test for sample weighted imagedatagen

* clarified docs + PEP8

* pep8 - blank line

* sample_weight argument after shuffle

* RNN docstring fixes; pass `maximum_iteration` argument to `while_loop` in TF.

* Use `embedding_lookup` for `gather` in TF backend (enables partitioned variables).

* Add `data_format` argument to Conv1D; docstring style fixes.

* Further fixes.

* Fix initializers (#9963)

* fix VarianceScaling

* Correct standard initializer testing

* Add test for truncated normal in backend

* Use correct theano function for truncated_normal

* Fix sloppiness in correction

* fix PEP

* improve style initialiser fix

* fix line length

* Use NCW/NWC for conv1d data format in TF backend.

* Model loading: do not return inside `try` block.

* Rely on `Flatten` layer to do data_format conversion in ResNet50, VGG16, VGG19.

* Remove potentially misleading warning messages in `applications`.

* Fix typo.

* Remove useless super delegation.

* Remove batch_size when doing stepwise fit (#10144)

`batch_size` is always `None` , because `steps_per_epoch` and `batch_size` are mutually exclusive.
Passing batch_size seems irrelevant as `None` is its default value in `test_loop`

* Add exceptions for `fit_loop` (#10145)

* Style fixes (#10148)

* Fix ConvLSTM2D unit_forget_bias initializer

* Avoid a warning by using  'x = x * y' instead of 'x *= y' (#10158)

* Fix undefined behaviour: preprocess_input copying/not copying the input arrays (#10153)

* Add copy option for image preprocessing

* Fix unnecessary import

* Fix style

* fix test error

* Make modifications in-place instead

* vae examples fixes (#10062)

* vae examples fixes

* white spaces around op

* comments and docstring fixes

* comments and docstring fixes

* comments and docstring fixes

* fixes on docs

* docs and spacing

* Merge 2 functions together in applications MobileNetV2 (#10163)

* merge 2 functions together

_inverted_res_block() and _first_inverted_res_block() are nearly the same. Merged them into one function
1. skip "Expand" part for 0 block
2. made layers names similar to original TF graph names

* added "mobl_" to prefix, pep8 fix

* remove dropout parameter

* concise prefix name

* Remove named argument from schedule function (#10178)

* Add missing doc (#10188)

* Best to set self.built=True at the end of build() (#10191)

When using Keras on TensorFlow (in graph mode), via `tf.keras`, then calling `self.add_weight()` fails if `self.built==True`, so it is best to encourage users to set `self.built=True` at the *end* of their `build()` method, rather than just "somewhere".

* Clean up preprocessing of `depthwise_kernel` for Theano (#10189)

* Allow dynamic backends in _need_convert_kernel (#10111)

* Fix _need_convert_kernel for external backends

* Don't assume external backend won't support NASNet

* Update from review comments

* fix TensorBoard callback with unit test (#10173)

* Allow project specific config files in Keras (#10183)

* Allow project specific config files in Keras

* Clearer code, updated comment

* Update cifar10_resnet.py (#10199)

change the comments on line 133.

* Adds height_shift_range to preprocessing.image doc and adds support for indentation in auto-generated doc (#10194)

* Update README.me

* Mobilenetv2 explanation  (#10174)

* i add mobilenetv2 to the table

* dd explanation for mobilenetv2

* add explanation for mibilenetv2

* Make multi_gpu_model serializable.

* Skip tests for multi gpu

* Fix slice namespace collision in backend.

* Skip multi_gpu tests for other backends than TF

* Replace np.ceil() with a faster operation (#10184)

* Replace ceil() with faster a operation

Replacing np.ceil() with faster operations, as suggested in #8488

* Revert "Replace ceil() with faster a operation"

This reverts commit 74fe60b445377b4f1be1d0a36b4777d7ed4f1c1b.

* Replace np.ceil() with a faster operation

* Fix syntax error

* Remove trailing whitespace

* Explicitly mention using a Sequence object in val (#10223)

* Grammatical error - "inputs channels" rather than "input". (#10217)

* Fix duplicated argname: num_gpus === parts (#10228)

Signed-off-by: CUI Wei <ghostplant@qq.com>

* Fix NASNet (#10209)

* Fix NASNet

* Update weight files

* In-place split to avoid inter-device duplication (#10230)

New Benchmark by in-place split:

>> keras.application.Resnet50 224x224x3 (NCWH; NVidia Tesla P100 x 4)
 input_shape = 3x224x224, batch_size =  96 x 4: 392(images/sec) => 417(images/sec)
 input_shape = 3x299x299, batch_size =  64 x 4: 229(images/sec) => 244(images/sec)
 input_shape = 3x224x224, batch_size =   8 x 4: 148(images/sec) => 163(images/sec)

>> keras.application.InceptionV3 (NCWH; NVidia Tesla P100 x 4)
 input_shape = 3x224x224, batch_size = 128 x 4: 488(images/sec) => 526(images/sec)
 input_shape = 3x299x299, batch_size =  96 x 4: 270(images/sec) => 294(images/sec)
 input_shape = 3x224x224, batch_size =   8 x 4: 146(images/sec) => 158(images/sec)

Signed-off-by: CUI Wei <ghostplant@qq.com>

* Increase test coverages by adding invalid CNTK usecases (#10236)

* Remove Sequential.model deprecation warning (#10256)

* Remove Sequential.model deprecation warning

* Remove dead line of code

* Increase test coverages by factorizing CNTK pads (#10259)

* Refactor ImageDataGenerator (#10130)

* Create get_random_transform and refactor

* Fix style and add tests

* Add more tests

* Fix documentation error

* Fix documentation style issue

* add apply_affine_transform

* document transformation dictionary

* Doc style fix

* Remove deprecated model.model from engine/saving (#10275)

* Typo in docstring for softplus (#10277)

Softplus docstring missing a parenthesis.

* Make Dot documentation inline with Concatenate (#10271)

Doc expects a list containing 2 tensors.

* Fixes automatic doc generation problem with nested lists. Adds a new test (#10212)

* Fixes automatic doc generation problem with indented lists. Adds a new test

* Some style fixes on doc automatic generation files

* Fixes a bad space in convolutional_recurrent.py

* Changes the test_doc_auto_generation in order to include a doc string taken from the codebase. Allows text lines following nested lists

* Use count_params function for non_trainable_count. (#10280)

* load_weights will fail if shape mismatch (#10266)

Fix for #10265

* Adds to and alphabetizes documentation of Layer base class. (#10282)

* Alphabetizes and adds to layers doc.

* Responding to @cais comments

* fix spacing.  Remove in(out)bound_nodes

* Non training Batch Norm operator has bad performance for it running into tensorflow's non fused batch norm API (#10207)

* When use tensorflow as backend, let batch norm run into fused batch norm as much as possible, which has better performance.

fix issue: http://github.com/keras-team/keras/issues/10058

* In Tensorflow backend, let batch norm call to FusedBatchNorm only NHWC format, also gamma and beta are not None.

Test result:
test env: with Tensorflow(commit a543d9471047ca3f6881c87105fcbe2cdff9207d Date:   Thu May 10 17:43:30 2018, local build), python3.4, centos7.4
test cases:
  "pytest  ./tests/keras/layers/normalization_test.py"  <all passed>
  "pytest  ./tests"      <keep same result as without this commit's modification on BN>

* fix code sytle.

* 1. Add axis parameter in backend's batch_normalization functions.
2. Refine the batch_normalization function in tensorflow backend, Let's it call to fused batch norm as much as possible.

Thanks the coments from fchollet.

* Trigger

* 1. add default value -1 for parameter axis in batch_normalization function in backend.
2. fix some code style.
Thanks the comments from fchollet.

* Handle capitalised extensions in list_pictures (#10220)

#10219

* Typo fix (#10293)

* Fix doc  (#10308)

* Fix naming convention

* Add missing doc

* Fix typo

* Improve docstrings of applications (#10310)

* Add pooling options in MobileNetV2 (#10313)

* Add pooling option

* Add pooling test

* Fix doc (#10327)

Fixed doc

* Handle `mask` in `TimeDistributed` wrapper. (#10242)

* equip TimeDistributed with mask and unspecified input length

* fix bugs in theano. add test on timedistributed + masking

* skip tests on cntk with multiple unspecified time lengths.

* move static shape inference to theano_backend, add docstring, etc.

* fix format

* Split `applications` and `preprocessing` modules. (#10339)

* Split `applications` and `preprocessing` modules.

* Fix dependencies.

* Move tests for applications (#10341)

* Improve the docstring of Conv3DTranspose (#10342)

* Add depth as third dimension in docstring of
  Conv3DTranspose in convolutional.py in keras.layers

* Reduce tests for applications (#10346)

* Reduce tests for applications

* Make selection over all models random

* Add an advanced activation layer for ReLU (#10322)

The max_value argument can not be used in a layer, except
custom layer or Lambda. Hence, similarly to LeakyReLU or
for example Softmax, this PR adds a layer for ReLU,
enabling also a capped ReLU to be used.

* FIX: Tensorboard callback only supports logging Embeddings layer weights (#7766)

* Embed layer-outputs rather than layer-weights in TensorBoard callback

* Update docstring and allow multiple inputs

* Fix tests

* Renaming

* Set learning phase

* Compute embeddings in batches

* Pass embedding data explicitly

* Actually process embeddings in batches

* Allow multiple inputs and validate input data

* Add example

* Delete utils.py

* Revert uncorrectly resolved merge-conflict

* Minor renaming

* Add comment clarifying the design choice

*  Fix HDF5Matrix issue when working in conjunction with TimeSeriesGenerator (#10334)

* Fix issue when working in conjunction with TimeSeriesGenerator

The TimeSeriesGenerator class uses xrange through six which caused an IndexError

* Add test

* Add corresponding test

* Fix for python3

* Simplified code

* Fix indent

* Fix test

* Supporting channels_first data format with crossentropy losses (#9715)

* Add error message when calling `summary` on unbuilt subclassed models.

* Prepare 2.2.0 release.

* Fix a version number (#10361)

* Update to Keras Applications 1.0.2 (fixes NASNet issue).

* Add tests for inputs set dynamically (#10367)

* CuDNN RNN layers nested in TimeDistributed are not converted when loading (#10357)

* Add a unit test for CuDNNGRU conversion with TimeDistributed.

* Extract duplicated function convert_model() to _convert_model_weights().

* #10356 Convert weights of CuDNN/plain RNN nested in TimeDistributed.

Same case as for Bidirectional, except that in TimeDistributed there's only
one nested layer instead of two.

* Style fix

* Update docs for 2.2.0.

* Add spatial dropout and 3D global pooling to docs (#10373)

* spatial dropout in docs

* 3d global pooling in docs

* Doc update (#10376)

A couple of variables are "used" in two examples without being defined. For consistency with other examples where auxiliary dimensions are defined, I think it would be better to explicitly assign them a value. I just used made up values, feel free to change to whatever makes more sense!

* Preserve input shape data when serializing deferred-build Sequential models.

* Add MXNet Backend (#59)

* Adding MXNet backend template. Adding all basic Variable and Tensor operations (#1)

* add activation functions

* add activation functions

* fix some legacy

* fix some legacy

* cross entropy

* cross entropy

* fix name scoping introduced in 2.0

* fix name scoping introduced in 2.0

* Add dropout, l2_normalization, random_normal/uniform/binomial (#2)

* remove the logic for hacking RNN

* remove the logic for hacking RNN

* add pooling with utils

* add pooling with utils

* minor

* lint and name scope fix

* fix access protected var

* fix add neighbor, removed __eq__ in KerasSymbol

* fix eval function, unittest for placeholder and variable

* add unittests

* fix bug

* fix bug

* fix

* add some temporary fixes in mxnet backend. undo change to the pytest.ini

* mxnet_backend graph fix, layer support  (#3)

* add activation functions

* fix some legacy

* cross entropy

* fix name scoping introduced in 2.0

* Add dropout, l2_normalization, random_normal/uniform/binomial (#2)

* remove the logic for hacking RNN

* add pooling with utils

* add activation functions

* fix some legacy

* cross entropy

* fix name scoping introduced in 2.0

* remove the logic for hacking RNN

* add pooling with utils

* minor

* lint and name scope fix

* fix access protected var

* fix add neighbor, removed __eq__ in KerasSymbol

* fix eval function, unittest for placeholder and variable

* add unittests

* fix bug

* fix bug

* fix

* add some temporary fixes in mxnet backend. undo change to the pytest.ini

* Keras function not working is a known issue, add skip in the test

* fix random_uniform/constant

* fix legacy randomize methods

* Fix MXNet backend operator bugs. Enabled Keras backend tests

* add bias

* Add Amazon copyrights to License (#6)

* fix

* fix

* fix backend for mlp

* fix context management, add optimizers

* minor change

* undo changes on example

* fix eval

* minor cleanup

* fix some property usage

* fixing AlphaDroupout, not finished yet

* add mx model instantiate

* modifies training model construct logic, fix some tests. fix reshape layer.

* minor fix

* fix bias_add

* more fix on Dense and bias_add

* In progress commit

* fix comment

* small fix

* remove pytest.skip in conv3d. But it failed with theano backend in my workspace though.

* Add conv2d and in_topk operator for mxnet backend (#11)

* Skip BatchDot tests for Theano backend. (#12)

* BatchDot, Basic Batchnorm, Fix BiasAdd, Fix Conv2D, CodeCleanup (#14)

* Fix Conv2d shape issues and enable Conv2D UTs

* Remove redundant mxnet only unit tests

* Adding batch_dot, remove deconv, code comments and cleanup

* Remove buggy conv1d implementation

* Fix CR comments. Fix lint check issues

* Move mxnet specific code from keras engine to mxnet_backend. (#15)

* Move MXNet optimizers from keras optimizers to mxnet backend (#16)

* Fix bug in reshape. Minor rename to avoid local conflicts

* Bug fixes and enable/skip all Keras tests for mxnet backend (#21)

* test results - 374 passed, 235 skipped in 114.44 seconds

* fix/skip keras tests - tests/integration_tests, tests/keras/applications

* fix/skip keras tests - tests/keras/engine/test_topology

* fix/skip keras tests - tests/keras/engine/test_training

* fix/skip keras tests - tests/keras/legacy/

* fix/skip keras tests - tests/keras/preprocessing

* fix/skip keras tests - tests/keras/utils/

* Fix CR comments

* Fix issues in zero_padding. Fix/Enable tests/layers/convolutional_test

* Add momentum to batchnorm. Enable/skip tests in layers/core, local, merge, noise, normalization

* Skip RNN tests in keras/tests/layers/recurrent_test, wrappers_test

* Fix bug in spatial padding, enable/skip tests in loss,optimizers,callback,loss_weighting, model_saving

* Fix mxnet backend multi-gpu training (#31)

Fixing bug for mxnet backend to use multiple gpus.

* Fix performance issue - Batchnormalization, Conv operator (#35)

* Fix default axis for batchnorm layer for channels_first data_format

* Performance improvement by avoiding kernel transpose in conv operation for channels_first format

* Fix model - architecture, weights and both, load and save. (#36)

* Prepare initial version of mxnet related documentation in keras (#38)

* Skip failing unit tests for unsupported functionality in mxnet backend

* Fix pep tests reported by CI

* Use pytest module skip, revert kernel_shape logic

* remove data_format param from bias_add API

* Allow Predict() without compile for mxnet backend and enable tests.

contributor - roywei@

* Fix bug - mxnet backend should not override keras config data_format to channels_first. Only warn of low performance

* Conv3d() operator implementation for Keras2.0 using MXNet backend (#40)

* conv3d implementation for keras2.0 as MXNet backend

* conv3d implementation/testing for keras2.0 using MXNet backend

* keeping -n option in pytest.ini file

* fixed comments given by Sandeep

* Add Conv1D support for MXNet backend (#44)

* Add Conv1D support for MXNet backend

* Fix CR comments

* Conv2d transpose (#47)

* add conv2d_transpose

* conv2d transpose for both channels, enabled test case

* add detailed comments and examples, fix style issue

* enable test case in topology

* Enable performance optimization for conv operators with MXNet backend. Make MXNet default backend with this branch (#48)

* Fix conv kernel shape bug for TF backend. (#50)

* Add support for keras multi_gpu_model() API with MXNet backend (#49)

* Add support for keras multi_gpu_model() API with MXNet backend. Autoset GPU0 context on GPU machine

* Fix typo

* Add SAME padding mode support for pooling operator. (#51)

* Add rnn() operator for MXNet backend with unrolling and masking feature (#46)

* Adding rnn() operator in Keras2.0 with MXNet as backend with unroll=True and Masking=True/False and enabled relevant testcases. Also, modified couple of operators.

* Modified comments

* Added comments to a method

* Enable categorical crossentropy testcases and made minor changes

* Modified message

* nit

* Added detail description of handling variable length input in RNN

* Skip conv2d_transpose and conv3d_transpose test-case for MXNet backend and minor changes in rnn()

* Adamax and NAdam optimizer for MXNet backend (#54)

* Add Adamax optimizer for MXNet backend

* Fix lr and adamax params

* Add Nadam optimizer for mxnet backend

* Add Conv3d transpose (#52)

* conv3d tranpose, enabled test case

* update kernel shape

* replace conv2d_transpse conv3d_transpose with convnd_transpose

* update value errors with MXNet Backend info, fix typo

* add check for conv3d transpose only supports gpu with cudnn

* update context check

* diable conv3d transpose test

* fix typo in comment

* Adding MXNet backend template. Adding all basic Variable and Tensor operations (#1)

* add activation functions

* add activation functions

* fix some legacy

* fix some legacy

* cross entropy

* cross entropy

* fix name scoping introduced in 2.0

* fix name scoping introduced in 2.0

* Add dropout, l2_normalization, random_normal/uniform/binomial (#2)

* remove the logic for hacking RNN

* remove the logic for hacking RNN

* add pooling with utils

* add pooling with utils

* minor

* lint and name scope fix

* fix access protected var

* fix add neighbor, removed __eq__ in KerasSymbol

* fix eval function, unittest for placeholder and variable

* add unittests

* fix bug

* fix bug

* fix

* add some temporary fixes in mxnet backend. undo change to the pytest.ini

* mxnet_backend graph fix, layer support  (#3)

* add activation functions

* fix some legacy

* cross entropy

* fix name scoping introduced in 2.0

* Add dropout, l2_normalization, random_normal/uniform/binomial (#2)

* remove the logic for hacking RNN

* add pooling with utils

* add activation functions

* fix some legacy

* cross entropy

* fix name scoping introduced in 2.0

* remove the logic for hacking RNN

* add pooling with utils

* minor

* lint and name scope fix

* fix access protected var

* fix add neighbor, removed __eq__ in KerasSymbol

* fix eval function, unittest for placeholder and variable

* add unittests

* fix bug

* fix bug

* fix

* add some temporary fixes in mxnet backend. undo change to the pytest.ini

* Keras function not working is a known issue, add skip in the test

* fix random_uniform/constant

* fix legacy randomize methods

* Fix MXNet backend operator bugs. Enabled Keras backend tests

* add bias

* Add Amazon copyrights to License (#6)

* fix

* fix

* fix backend for mlp

* fix context management, add optimizers

* minor change

* undo changes on example

* fix eval

* minor cleanup

* fix some property usage

* fixing AlphaDroupout, not finished yet

* add mx model instantiate

* modifies training model construct logic, fix some tests. fix reshape layer.

* minor fix

* fix bias_add

* more fix on Dense and bias_add

* In progress commit

* fix comment

* small fix

* remove pytest.skip in conv3d. But it failed with theano backend in my workspace though.

* Add conv2d and in_topk operator for mxnet backend (#11)

* Skip BatchDot tests for Theano backend. (#12)

* BatchDot, Basic Batchnorm, Fix BiasAdd, Fix Conv2D, CodeCleanup (#14)

* Fix Conv2d shape issues and enable Conv2D UTs

* Remove redundant mxnet only unit tests

* Adding batch_dot, remove deconv, code comments and cleanup

* Remove buggy conv1d implementation

* Fix CR comments. Fix lint check issues

* Move mxnet specific code from keras engine to mxnet_backend. (#15)

* Move MXNet optimizers from keras optimizers to mxnet backend (#16)

* Fix bug in reshape. Minor rename to avoid local conflicts

* Bug fixes and enable/skip all Keras tests for mxnet backend (#21)

* test results - 374 passed, 235 skipped in 114.44 seconds

* fix/skip keras tests - tests/integration_tests, tests/keras/applications

* fix/skip keras tests - tests/keras/engine/test_topology

* fix/skip keras tests - tests/keras/engine/test_training

* fix/skip keras tests - tests/keras/legacy/

* fix/skip keras tests - tests/keras/preprocessing

* fix/skip keras tests - tests/keras/utils/

* Fix CR comments

* Fix issues in zero_padding. Fix/Enable tests/layers/convolutional_test

* Add momentum to batchnorm. Enable/skip tests in layers/core, local, merge, noise, normalization

* Skip RNN tests in keras/tests/layers/recurrent_test, wrappers_test

* Fix bug in spatial padding, enable/skip tests in loss,optimizers,callback,loss_weighting, model_saving

* Fix mxnet backend multi-gpu training (#31)

Fixing bug for mxnet backend to use multiple gpus.

* Fix performance issue - Batchnormalization, Conv operator (#35)

* Fix default axis for batchnorm layer for channels_first data_format

* Performance improvement by avoiding kernel transpose in conv operation for channels_first format

* Fix model - architecture, weights and both, load and save. (#36)

* Prepare initial version of mxnet related documentation in keras (#38)

* Skip failing unit tests for unsupported functionality in mxnet backend

* Fix pep tests reported by CI

* Use pytest module skip, revert kernel_shape logic

* remove data_format param from bias_add API

* Allow Predict() without compile for mxnet backend and enable tests.

contributor - roywei@

* Fix bug - mxnet backend should not override keras config data_format to channels_first. Only warn of low performance

* Conv3d() operator implementation for Keras2.0 using MXNet backend (#40)

* conv3d implementation for keras2.0 as MXNet backend

* conv3d implementation/testing for keras2.0 using MXNet backend

* keeping -n option in pytest.ini file

* fixed comments given by Sandeep

* Add Conv1D support for MXNet backend (#44)

* Add Conv1D support for MXNet backend

* Fix CR comments

* Conv2d transpose (#47)

* add conv2d_transpose

* conv2d transpose for both channels, enabled test case

* add detailed comments and examples, fix style issue

* enable test case in topology

* Enable performance optimization for conv operators with MXNet backend. Make MXNet default backend with this branch (#48)

* Fix conv kernel shape bug for TF backend. (#50)

* Add support for keras multi_gpu_model() API with MXNet backend (#49)

* Add support for keras multi_gpu_model() API with MXNet backend. Autoset GPU0 context on GPU machine

* Fix typo

* Add SAME padding mode support for pooling operator. (#51)

* Add rnn() operator for MXNet backend with unrolling and masking feature (#46)

* Adding rnn() operator in Keras2.0 with MXNet as backend with unroll=True and Masking=True/False and enabled relevant testcases. Also, modified couple of operators.

* Modified comments

* Added comments to a method

* Enable categorical crossentropy testcases and made minor changes

* Modified message

* nit

* Added detail description of handling variable length input in RNN

* Skip conv2d_transpose and conv3d_transpose test-case for MXNet backend and minor changes in rnn()

* Adamax and NAdam optimizer for MXNet backend (#54)

* Add Adamax optimizer for MXNet backend

* Fix lr and adamax params

* Add Nadam optimizer for mxnet backend

* Add Conv3d transpose (#52)

* conv3d tranpose, enabled test case

* update kernel shape

* replace conv2d_transpse conv3d_transpose with convnd_transpose

* update value errors with MXNet Backend info, fix typo

* add check for conv3d transpose only supports gpu with cudnn

* update context check

* diable conv3d transpose test

* fix typo in comment

* Rebase to latest Keras - April 3, 2018

* Add build badges

* Fix multi_gpu API bug for CPU. Fix PEP. (#64)

* Fix multi_gpu API bug for CPU. Fix PEP.

* fix embedding layer bug (#61)

* fix embedding bug

* addressed comments, enabled more test cases

* add keras test

* reduce line length

* fix style, add blank lines

* Benchmark (#55)

* add conv2d_transpose

* conv2d transpose for both channels, enabled test case

* add detailed comments and examples, fix style issue

* add benchmark scripts for resnet and imagenet data

* combine scripts

* fix args

* fix num of gpus

* update log

* multi_gpu_model only support tf

* add benchamrk scripts for synthetic data

* update read me and scripts

* add mxnet traing result table

* update on readme

* add cifar10 dataset and enable various resnet layers

* fix compile for mxnet multiple gpu

* update callbacks

* update synthetic data script, add credits

* undo new line

* update readme, addressed pr comments

* update readme

* benchmark scripts style fix (#66)

* style fix

* remove unused import, fix line too long

* adrressed pr comments

* Added keras util API for conversion of data tensor from channels_last to channels_first using MXNet backend (#65)

* Added keras util API for conversion of data tensor from channels_last to channels_first using MXNet backend

* Modified comments

* Addressed review comments and made the API more generic accross backends

* Removed shape check

* Modified comments

* Added edge cases

* moved helper method as nested

* Added RNN benchmark scripts (#69)

* Added RNN benchmark scripts

* Fixed new line in bash script

* Removed different backend code and modified comments

* Removed spacing

* Automated the wikiText2 download script

* Added dataset_util functionality to have more flexible code

* Added minor comments

* modified minor comments

* Fixed the multi-gpu context (#68)

* Update benchmark result (#70)

* update benchmark result

* update result

* simplify folder structure

* add image result

* add note

* add note

* Add MXNet Backend (#59)

* Adding MXNet backend template. Adding all basic Variable and Tensor operations (#1)

* add activation functions

* add activation functions

* fix some legacy

* fix some legacy

* cross entropy

* cross entropy

* fix name scoping introduced in 2.0

* fix name scoping introduced in 2.0

* Add dropout, l2_normalization, random_normal/uniform/binomial (#2)

* remove the logic for hacking RNN

* remove the logic for hacking RNN

* add pooling with utils

* add pooling with utils

* minor

* lint and name scope fix

* fix access protected var

* fix add neighbor, removed __eq__ in KerasSymbol

* fix eval function, unittest for placeholder and variable

* add unittests

* fix bug

* fix bug

* fix

* add some temporary fixes in mxnet backend. undo change to the pytest.ini

* mxnet_backend graph fix, layer support  (#3)

* add activation functions

* fix some legacy

* cross entropy

* fix name scoping introduced in 2.0

* Add dropout, l2_normalization, random_normal/uniform/binomial (#2)

* remove the logic for hacking RNN

* add pooling with utils

* add activation functions

* fix some legacy

* cross entropy

* fix name scoping introduced in 2.0

* remove the logic for hacking RNN

* add pooling with utils

* minor

* lint and name scope fix

* fix access protected var

* fix add neighbor, removed __eq__ in KerasSymbol

* fix eval function, unittest for placeholder and variable

* add unittests

* fix bug

* fix bug

* fix

* add some temporary fixes in mxnet backend. undo change to the pytest.ini

* Keras function not working is a known issue, add skip in the test

* fix random_uniform/constant

* fix legacy randomize methods

* Fix MXNet backend operator bugs. Enabled Keras backend tests

* add bias

* Add Amazon copyrights to License (#6)

* fix

* fix

* fix backend for mlp

* fix context management, add optimizers

* minor change

* undo changes on example

* fix eval

* minor cleanup

* fix some property usage

* fixing AlphaDroupout, not finished yet

* add mx model instantiate

* modifies training model construct logic, fix some tests. fix reshape layer.

* minor fix

* fix bias_add

* more fix on Dense and bias_add

* In progress commit

* fix comment

* small fix

* remove pytest.skip in conv3d. But it failed with theano backend in my workspace though.

* Add conv2d and in_topk operator for mxnet backend (#11)

* Skip BatchDot tests for Theano backend. (#12)

* BatchDot, Basic Batchnorm, Fix BiasAdd, Fix Conv2D, CodeCleanup (#14)

* Fix Conv2d shape issues and enable Conv2D UTs

* Remove redundant mxnet only unit tests

* Adding batch_dot, remove deconv, code comments and cleanup

* Remove buggy conv1d implementation

* Fix CR comments. Fix lint check issues

* Move mxnet specific code from keras engine to mxnet_backend. (#15)

* Move MXNet optimizers from keras optimizers to mxnet backend (#16)

* Fix bug in reshape. Minor rename to avoid local conflicts

* Bug fixes and enable/skip all Keras tests for mxnet backend (#21)

* test results - 374 passed, 235 skipped in 114.44 seconds

* fix/skip keras tests - tests/integration_tests, tests/keras/applications

* fix/skip keras tests - tests/keras/engine/test_topology

* fix/skip keras tests - tests/keras/engine/test_training

* fix/skip keras tests - tests/keras/legacy/

* fix/skip keras tests - tests/keras/preprocessing

* fix/skip keras tests - tests/keras/utils/

* Fix CR comments

* Fix issues in zero_padding. Fix/Enable tests/layers/convolutional_test

* Add momentum to batchnorm. Enable/skip tests in layers/core, local, merge, noise, normalization

* Skip RNN tests in keras/tests/layers/recurrent_test, wrappers_test

* Fix bug in spatial padding, enable/skip tests in loss,optimizers,callback,loss_weighting, model_saving

* Fix mxnet backend multi-gpu training (#31)

Fixing bug for mxnet backend to use multiple gpus.

* Fix performance issue - Batchnormalization, Conv operator (#35)

* Fix default axis for batchnorm layer for channels_first data_format

* Performance improvement by avoiding kernel transpose in conv operation for channels_first format

* Fix model - architecture, weights and both, load and save. (#36)

* Prepare initial version of mxnet related documentation in keras (#38)

* Skip failing unit tests for unsupported functionality in mxnet backend

* Fix pep tests reported by CI

* Use pytest module skip, revert kernel_shape logic

* remove data_format param from bias_add API

* Allow Predict() without compile for mxnet backend and enable tests.

contributor - roywei@

* Fix bug - mxnet backend should not override keras config data_format to channels_first. Only warn of low performance

* Conv3d() operator implementation for Keras2.0 using MXNet backend (#40)

* conv3d implementation for keras2.0 as MXNet backend

* conv3d implementation/testing for keras2.0 using MXNet backend

* keeping -n option in pytest.ini file

* fixed comments given by Sandeep

* Add Conv1D support for MXNet backend (#44)

* Add Conv1D support for MXNet backend

* Fix CR comments

* Conv2d transpose (#47)

* add conv2d_transpose

* conv2d transpose for both channels, enabled test case

* add detailed comments and examples, fix style issue

* enable test case in topology

* Enable performance optimization for conv operators with MXNet backend. Make MXNet default backend with this branch (#48)

* Fix conv kernel shape bug for TF backend. (#50)

* Add support for keras multi_gpu_model() API with MXNet backend (#49)

* Add support for keras multi_gpu_model() API with MXNet backend. Autoset GPU0 context on GPU machine

* Fix typo

* Add SAME padding mode support for pooling operator. (#51)

* Add rnn() operator for MXNet backend with unrolling and masking feature (#46)

* Adding rnn() operator in Keras2.0 with MXNet as backend with unroll=True and Masking=True/False and enabled relevant testcases. Also, modified couple of operators.

* Modified comments

* Added comments to a method

* Enable categorical crossentropy testcases and made minor changes

* Modified message

* nit

* Added detail description of handling variable length input in RNN

* Skip conv2d_transpose and conv3d_transpose test-case for MXNet backend and minor changes in rnn()

* Adamax and NAdam optimizer for MXNet backend (#54)

* Add Adamax optimizer for MXNet backend

* Fix lr and adamax params

* Add Nadam optimizer for mxnet backend

* Add Conv3d transpose (#52)

* conv3d tranpose, enabled test case

* update kernel shape

* replace conv2d_transpse conv3d_transpose with convnd_transpose

* update value errors with MXNet Backend info, fix typo

* add check for conv3d transpose only supports gpu with cudnn

* update context check

* diable conv3d transpose test

* fix typo in comment

* Adding MXNet backend template. Adding all basic Variable and Tensor operations (#1)

* add activation functions

* add activation functions

* fix some legacy

* fix some legacy

* cross entropy

* cross entropy

* fix name scoping introduced in 2.0

* fix name scoping introduced in 2.0

* Add dropout, l2_normalization, random_normal/uniform/binomial (#2)

* remove the logic for hacking RNN

* remove the logic for hacking RNN

* add pooling with utils

* add pooling with utils

* minor

* lint and name scope fix

* fix access protected var

* fix add neighbor, removed __eq__ in KerasSymbol

* fix eval function, unittest for placeholder and variable

* add unittests

* fix bug

* fix bug

* fix

* add some temporary fixes in mxnet backend. undo change to the pytest.ini

* mxnet_backend graph fix, layer support  (#3)

* add activation functions

* fix some legacy

* cross entropy

* fix name scoping introduced in 2.0

* Add dropout, l2_normalization, random_normal/uniform/binomial (#2)

* remove the logic for hacking RNN

* add pooling with utils

* add activation functions

* fix some legacy

* cross entropy

* fix name scoping introduced in 2.0

* remove the logic for hacking RNN

* add pooling with utils

* minor

* lint and name scope fix

* fix access protected var

* fix add neighbor, removed __eq__ in KerasSymbol

* fix eval function, unittest for placeholder and variable

* add unittests

* fix bug

* fix bug

* fix

* add some temporary fixes in mxnet backend. undo change to the pytest.ini

* Keras function not working is a known issue, add skip in the test

* fix random_uniform/constant

* fix legacy randomize methods

* Fix MXNet backend operator bugs. Enabled Keras backend tests

* add bias

* Add Amazon copyrights to License (#6)

* fix

* fix

* fix backend for mlp

* fix context management, add optimizers

* minor change

* undo changes on example

* fix eval

* minor cleanup

* fix some property usage

* fixing AlphaDroupout, not finished yet

* add mx model instantiate

* modifies training model construct logic, fix some tests. fix reshape layer.

* minor fix

* fix bias_add

* more fix on Dense and bias_add

* In progress commit

* fix comment

* small fix

* remove pytest.skip in conv3d. But it failed with theano backend in my workspace though.

* Add conv2d and in_topk operator for mxnet backend (#11)

* Skip BatchDot tests for Theano backend. (#12)

* BatchDot, Basic Batchnorm, Fix BiasAdd, Fix Conv2D, CodeCleanup (#14)

* Fix Conv2d shape issues and enable Conv2D UTs

* Remove redundant mxnet only unit tests

* Adding batch_dot, remove deconv, code comments and cleanup

* Remove buggy conv1d implementation

* Fix CR comments. Fix lint check issues

* Move mxnet specific code from keras engine to mxnet_backend. (#15)

* Move MXNet optimizers from keras optimizers to mxnet backend (#16)

* Fix bug in reshape. Minor rename to avoid local conflicts

* Bug fixes and enable/skip all Keras tests for mxnet backend (#21)

* test results - 374 passed, 235 skipped in 114.44 seconds

* fix/skip keras tests - tests/integration_tests, tests/keras/applications

* fix/skip keras tests - tests/keras/engine/test_topology

* fix/skip keras tests - tests/keras/engine/test_training

* fix/skip keras tests - tests/keras/legacy/

* fix/skip keras tests - tests/keras/preprocessing

* fix/skip keras tests - tests/keras/utils/

* Fix CR comments

* Fix issues in zero_padding. Fix/Enable tests/layers/convolutional_test

* Add momentum to batchnorm. Enable/skip tests in layers/core, local, merge, noise, normalization

* Skip RNN tests in keras/tests/layers/recurrent_test, wrappers_test

* Fix bug in spatial padding, enable/skip tests in loss,optimizers,callback,loss_weighting, model_saving

* Fix mxnet backend multi-gpu training (#31)

Fixing bug for mxnet backend to use multiple gpus.

* Fix performance issue - Batchnormalization, Conv operator (#35)

* Fix default axis for batchnorm layer for channels_first data_format

* Performance improvement by avoiding kernel transpose in conv operation for channels_first format

* Fix model - architecture, weights and both, load and save. (#36)

* Prepare initial version of mxnet related documentation in keras (#38)

* Skip failing unit tests for unsupported functionality in mxnet backend

* Fix pep tests reported by CI

* Use pytest module skip, revert kernel_shape logic

* remove data_format param from bias_add API

* Allow Predict() without compile for mxnet backend and enable tests.

contributor - roywei@

* Fix bug - mxnet backend should not override keras config data_format to channels_first. Only warn of low performance

* Conv3d() operator implementation for Keras2.0 using MXNet backend (#40)

* conv3d implementation for keras2.0 as MXNet backend

* conv3d implementation/testing for keras2.0 using MXNet backend

* keeping -n option in pytest.ini file

* fixed comments given by Sandeep

* Add Conv1D support for MXNet backend (#44)

* Add Conv1D support for MXNet backend

* Fix CR comments

* Conv2d transpose (#47)

* add conv2d_transpose

* conv2d transpose for both channels, enabled test case

* add detailed comments and examples, fix style issue

* enable test case in topology

* Enable performance optimization for conv operators with MXNet backend. Make MXNet default backend with this branch (#48)

* Fix conv kernel shape bug for TF backend. (#50)

* Add support for keras multi_gpu_model() API with MXNet backend (#49)

* Add support for keras multi_gpu_model() API with MXNet backend. Autoset GPU0 context on GPU machine

* Fix typo

* Add SAME padding mode support for pooling operator. (#51)

* Add rnn() operator for MXNet backend with unrolling and masking feature (#46)

* Adding rnn() operator in Keras2.0 with MXNet as backend with unroll=True and Masking=True/False and enabled relevant testcases. Also, modified couple of operators.

* Modified comments

* Added comments to a method

* Enable categorical crossentropy testcases and made minor changes

* Modified message

* nit

* Added detail description of handling variable length input in RNN

* Skip conv2d_transpose and conv3d_transpose test-case for MXNet backend and minor changes in rnn()

* Adamax and NAdam optimizer for MXNet backend (#54)

* Add Adamax optimizer for MXNet backend

* Fix lr and adamax params

* Add Nadam optimizer for mxnet backend

* Add Conv3d transpose (#52)

* conv3d tranpose, enabled test case

* update kernel shape

* replace conv2d_transpse conv3d_transpose with convnd_transpose

* update value errors with MXNet Backend info, fix typo

* add check for conv3d transpose only supports gpu with cudnn

* update context check

* diable conv3d transpose test

* fix typo in comment

* Rebase to latest Keras - April 3, 2018

* Add build badges

* Fix multi_gpu API bug for CPU. Fix PEP. (#64)

* Fix multi_gpu API bug for CPU. Fix PEP.

* fix embedding layer bug (#61)

* fix embedding bug

* addressed comments, enabled more test cases

* add keras test

* reduce line length

* fix style, add blank lines

* Benchmark (#55)

* add conv2d_transpose

* conv2d transpose for both channels, enabled test case

* add detailed comments and examples, fix style issue

* add benchmark scripts for resnet and imagenet data

* combine scripts

* fix args

* fix num of gpus

* update log

* multi_gpu_model only support tf

* add benchamrk scripts for synthetic data

* update read me and scripts

* add mxnet traing result table

* update on readme

* add cifar10 dataset and enable various resnet layers

* fix compile for mxnet multiple gpu

* update callbacks

* update synthetic data script, add credits

* undo new line

* update readme, addressed pr comments

* update readme

* benchmark scripts style fix (#66)

* style fix

* remove unused import, fix line too long

* adrressed pr comments

* Added keras util API for conversion of data tensor from channels_last to channels_first using MXNet backend (#65)

* Added keras util API for conversion of data tensor from channels_last to channels_first using MXNet backend

* Modified comments

* Addressed review comments and made the API more generic accross backends

* Removed shape check

* Modified comments

* Added edge cases

* moved helper method as nested

* Added RNN benchmark scripts (#69)

* Added RNN benchmark scripts

* Fixed new line in bash script

* Removed different backend code and modified comments

* Removed spacing

* Automated the wikiText2 download script

* Added dataset_util functionality to have more flexible code

* Added minor comments

* modified minor comments

* Fixed the multi-gpu context (#68)

* Update benchmark result (#70)

* update benchmark result

* update result

* simplify folder structure

* add image result

* add note

* add note

* rebase to latest Keras - April 20, 2018, fix bug and unit tests

* Added detailed RNN results (#73)

* Added detailed RNN results

* Modified table content and added CUDA version

* fix keras examples (#72)

* fix auto encoder examples

* update other examples

* fix style and add ctc not implemented error

* Added Detailed RNN results (#77)

* Modified RNN benchmark document

* Added minor comments

* fixed broken image link

* Added API to extract metrics from a test and also added epoch parameter (#78)

* Add mxnet backend tutorial documents (#76)

* add performance tips document

* update warning

* add docs from wiki

* add initial multi gpu doc, simplified installation doc, fix benchmark doc typo

* update install steps

* add multi_gpu_model tutorial

* Support exporting model as MXNet model (sym, params). (#80)

* Support exporting model as MXNet model (sym, params).

* Return data_names and data_shapes

* add unit tests for mxnet model save API

* Add test with LSTM layer for mxnet model save API

* Add support for functional Model graphs in save_mxnet_model API

* Add additional logging for cnn benchmarks (#89)

* add extra logging

* add logging for cnn synthetic

* fix log name

* fix file name

* Log RNN benchmark results (#90)

* Make benchmark result logging available in RNN scripts

* Make log file name consistent across CNN and RNN benchmarks

* fix pytest errors (#93)

* Cherry pick keras-team/keras 2.1.6 missing 3 commits into awslabs/keras-apache-mxnet (#96)

* update multi_gpu api in benchmark scripts (#95)

* update multi_gpu

* update logging

* fix logging

* fix logging

* fix speed format

* remove learning rate log

* Revamp keras-mxnet docs (#82)

* Update main README and move mxnet_backend_docs under docs

* revisit installation mxnet backend docs

* revisit multi_gpu_training mxnet backend docs

* revisit performance_guide mxnet backend docs

* revisit using rnn with mxnet backend in mxnet backend docs

* add save_mxnet_model tutorials in mxnet backend docs

* Fixing review comments from aaron

* Resolve CR comments on save_mxnet_model tutorial

* Fix broken links, update tutorial links in the mxnet_backend code

* revamp benchmark results readme

* Benchmark results README page revamp

* Add library versions

* Remove too detailed benchmark results. Summarize in README

* Get back detailed results document

* Remove experiemental RNN benchmarks from README

* addressed review comments on benchmark results

* Set latest stable dependency of h5py to avoid warnings

* Rebase to latest Keras April 20 2018 (#71)

* Improve tests by designating dtype of sample data (#9834)

* Document that "same" is inconsistent across backends with strides!=1 (#9629)

* Document that `"same"` is inconsistent across backends with `strides` != 1

* Use "[here](...)"

* #9642 Add kwarg and documentation for dilation_rate to SeparableConvs (#9844)

* Add kwarg and documentation for dilation_rate to SeparableConvs

* Fix pep8 complaint

I forgot to check the style before committing. Pep8 was complaining about a missing whitespace after comma, now it's fixed.

* fit/evaluate_generator supporting native tensors (#9816)

Currently, `fit/evaluate_generator` don't support this case without this fix.
But framework-native data tensors are already supported by `_fit_loop` and `_test_loop`.

Signed-off-by: CUI Wei <ghostplant@qq.com>

* Add h5py to dependencies

* Fixed typo. (#9866)

* Fix image_ocr.py example ValueError (#9869)

* Fixed the NASNet issue. (#9865)

* Fixed the NASNet issue.

* Nasnet doesn't require flatten.

* Updated documentation accordingly.

* Removed generate dropout ones from recurrent. (#9892)

* Removed generate dropout ones from recurrent.

* Fixed index issue.

* Fix `in_test_phase` of CNTK and Add its tests (#9902)

* Fix dtype designation for `variable` of CNTK and Add its tests (#9903)

* import `pydot`, improve error messages about `pydot` and GraphViz, bump to `pydot >= 1.2.4` (#9904)

* REL: bump to `pydot >= 1.2.4` in `extras_require`

* MAI: import pydot (as required in `extras_require`)

* MAI: refine error messages for `pydot` and GraphViz

distinguish between absence of `pydot` and failure to find
the executables of GraphViz in the $PATH.

* DEV: ignore `.pytest_cache`

* Fix documentation of flow_from_directory() (#9910)

The way the documentation is parsed for the Keras website made some lines of the documentation beginning with "Default:" look funny. Also changed the documentation of return value to be clear that it always returns a batch of images.

* ModelCheckpoint: print previous best (#9911)

* multi_gpu_model supporting legacy/fullCPU/fullGPU (#9638)

Signed-off-by: CUI Wei <ghostplant@qq.com>

* Fix `batch_dot` of Theano when `axes=0` (#9920)

* Fix `batch_dot` of CNTK when `axes=None` (#9921)

* Fix `batch_dot` of TensorFlow when `axes=None` (#9922)

* Fix stateful metrics when passing dict to compile (#9894)

* Added note to manually install h5py where needed (#9830)

* Added notes to manually install h5py if needed

* Added FAQ entry on h5py

* deleted redundant remark about h5py

* updated FAQ to reflect dependency change

* fixed comment format to pass failing test

* removed new trailing whitespaces

* improved docstring format

* reverted callbacks.py

* fixed links in model.py

* updated faq.py

* link pointing to FAQ

* Add support for `constants` in Bidirectional wrapper (#9260)

* Add support fot `constants` in Bidirectional wrapper

* Add more tests for Bidirectional wrapper

* Fix `compute_mask` for Birectional with return_state=True

Fix `compute_mask` to properly support `return_state` introduced in Birectional with #8977

* Add test for Bidirectional with unknown timestamps

* Skip test for CNTK for unknown timestamps with Bidirectional

* avoid override the input constant when need broadcast sequential axis on rnn's constant

* Move _standardize_args to recurrent, remove duplication

* Fix  for Birectional when multiple masks are passed

* Updated for TF 1.7 (#9937)

* fix TimeSeriesGenerator glitch (#9899)

* Added an error message for undefined shape on NASNet. (#9891)

* Added an error message for undefined shape on NASNet.

* Forgot that the message should be present only when loading imagenet weights.

* Changed the message.

* Fix PEP8

* Allow shift_range to be 1-D array-like or int (#8869)

* Allow shift_range to be 1-D array-like or int

* Add docstrings

* Fix conflict resolution merge minor disaster

* remove stray line from merge

* Remove extra "tabs"

* Exclude multi-gpu utils when reporting coverages (#9942)

* Make conv_invalid_use and pooling_invalid_use efficient (#9944)

* Chenta/cntk bn (#9952)

* fix cntk static learning phase issue; add a test

* fix code style;add more comments

* add boolean support

* fix code style issue

* Immigrate reference operations to a separate module (#9948)

* Add MXNet Backend (#59)

* Adding MXNet backend template. Adding all basic Variable and Tensor operations (#1)

* add activation functions

* add activation functions

* fix some legacy

* fix some legacy

* cross entropy

* cross entropy

* fix name scoping introduced in 2.0

* fix name scoping introduced in 2.0

* Add dropout, l2_normalization, random_normal/uniform/binomial (#2)

* remove the logic for hacking RNN

* remove the logic for hacking RNN

* add pooling with utils

* add pooling with utils

* minor

* lint and name scope fix

* fix access protected var

* fix add neighbor, removed __eq__ in KerasSymbol

* fix eval function, unittest for placeholder and variable

* add unittests

* fix bug

* fix bug

* fix

* add some temporary fixes in mxnet backend. undo change to the pytest.ini

* mxnet_backend graph fix, layer support  (#3)

* add activation functions

* fix some legacy

* cross entropy

* fix name scoping introduced in 2.0

* Add dropout, l2_normalization, random_normal/uniform/binomial (#2)

* remove the logic for hacking RNN

* add pooling with utils

* add activation functions

* fix some legacy

* cross entropy

* fix name scoping introduced in 2.0

* remove the logic for hacking RNN

* add pooling with utils

* minor

* lint and name scope fix

* fix access protected var

* fix add neighbor, removed __eq__ in KerasSymbol

* fix eval function, unittest for placeholder and variable

* add unittests

* fix bug

* fix bug

* fix

* add some temporary fixes in mxnet backend. undo change to the pytest.ini

* Keras function not working is a known issue, add skip in the test

* fix random_uniform/constant

* fix legacy randomize methods

* Fix MXNet backend operator bugs. Enabled Keras backend tests

* add bias

* Add Amazon copyrights to License (#6)

* fix

* fix

* fix backend for mlp

* fix context management, add optimizers

* minor change

* undo changes on example

* fix eval

* minor cleanup

* fix some property usage

* fixing AlphaDroupout, not finished yet

* add mx model instantiate

* modifies training model construct logic, fix some tests. fix reshape layer.

* minor fix

* fix bias_add

* more fix on Dense and bias_add

* In progress commit

* fix comment

* small fix

* remove pytest.skip in conv3d. But it failed with theano backend in my workspace though.

* Add conv2d and in_topk operator for mxnet backend (#11)

* Skip BatchDot tests for Theano backend. (#12)

* BatchDot, Basic Batchnorm, Fix BiasAdd, Fix Conv2D, CodeCleanup (#14)

* Fix Conv2d shape issues and enable Conv2D UTs

* Remove redundant mxnet only unit tests

* Adding batch_dot, remove deconv, code comments and cleanup

* Remove buggy conv1d implementation

* Fix CR comments. Fix lint check issues

* Move mxnet specific code from keras engine to mxnet_backend. (#15)

* Move MXNet optimizers from keras optimizers to mxnet backend (#16)

* Fix bug in reshape. Minor rename to avoid local conflicts

* Bug fixes and enable/skip all Keras tests for mxnet backend (#21)

* test results - 374 passed, 235 skipped in 114.44 seconds

* fix/skip keras tests - tests/integration_tests, tests/keras/applications

* fix/skip keras tests - tests/keras/engine/test_topology

* fix/skip keras tests - tests/keras/engine/test_training

* fix/skip keras tests - tests/keras/legacy/

* fix/skip keras tests - tests/keras/preprocessing

* fix/skip keras tests - tests/keras/utils/

* Fix CR comments

* Fix issues in zero_padding. Fix/Enable tests/layers/convolutional_test

* Add momentum to batchnorm. Enable/skip tests in layers/core, local, merge, noise, normalization

* Skip RNN tests in keras/tests/layers/recurrent_test, wrappers_test

* Fix bug in spatial padding, enable/skip tests in loss,optimizers,callback,loss_weighting, model_saving

* Fix mxnet backend multi-gpu training (#31)

Fixing bug for mxnet backend to use multiple gpus.

* Fix performance issue - Batchnormalization, Conv operator (#35)

* Fix default axis for batchnorm layer for channels_first data_format

* Performance improvement by avoiding kernel transpose in conv operation for channels_first format

* Fix model - architecture, weights and both, load and save. (#36)

* Prepare initial version of mxnet related documentation in keras (#38)

* Skip failing unit tests for unsupported functionality in mxnet backend

* Fix pep tests reported by CI

* Use pytest module skip, revert kernel_shape logic

* remove data_format param from bias_add API

* Allow Predict() without compile for mxnet backend and enable tests.

contributor - royw…
sandeep-krishnamurthy added a commit to awslabs/keras-apache-mxnet that referenced this pull request Aug 14, 2018
* Document that "same" is inconsistent across backends with strides!=1 (#9629)

* Document that `"same"` is inconsistent across backends with `strides` != 1

* Use "[here](...)"

* Add h5py to dependencies

* import `pydot`, improve error messages about `pydot` and GraphViz, bump to `pydot >= 1.2.4` (#9904)

* REL: bump to `pydot >= 1.2.4` in `extras_require`

* MAI: import pydot (as required in `extras_require`)

* MAI: refine error messages for `pydot` and GraphViz

distinguish between absence of `pydot` and failure to find
the executables of GraphViz in the $PATH.

* DEV: ignore `.pytest_cache`

* Added note to manually install h5py where needed (#9830)

* Added notes to manually install h5py if needed

* Added FAQ entry on h5py

* deleted redundant remark about h5py

* updated FAQ to reflect dependency change

* fixed comment format to pass failing test

* removed new trailing whitespaces

* improved docstring format

* reverted callbacks.py

* fixed links in model.py

* updated faq.py

* link pointing to FAQ

* Added an error message for undefined shape on NASNet. (#9891)

* Added an error message for undefined shape on NASNet.

* Forgot that the message should be present only when loading imagenet weights.

* Changed the message.

* Fix PEP8

* Make conv_invalid_use and pooling_invalid_use efficient (#9944)

* Chenta/cntk bn (#9952)

* fix cntk static learning phase issue; add a test

* fix code style;add more comments

* add boolean support

* fix code style issue

* fixing typos (#10016)

* Add `separable_conv2d` for Theano (#10003)

* Add `separable_conv2d` for Theano

* Enable the layer test for `separable_conv2d`

* Fix the strides for 1x1 conv

* Refactor topological part of `engine` module (#10023)

* Refactor topological part of Keras engine.

* Fix imports

* Fix merge mixup.

* Fix Bidirectional Regularization (#10012)

* Fix Bidirectional Regularization

Override the Wrapper `get_updates_for` and `get_losses_for` methods so that contributions from both the forward and backward layers are included in Bidirectional.

* Use Parent for Calculating Inner Inputs

Remove unneeded inner input calculations.

* Simplify Bidirectional Losses

* Add Bidirectional Unit Tests

Test Bidirectional updates and losses.

* Remove Trailing Whitespace

* Fix Bidirectional Loss Inputs

* Add Tests for Conditional Updates/Losses

* Remove Whitespace

* Refactor training part of `engine` module. (#10029)

* Refactor topological part of Keras engine.

* Fix imports

* Fix merge mixup.

* Refactor training part of the Keras engine.

* Fix unit tests.

* Fix `int_shape` of Theano and Refactor associated lines (#10030)

* Add exceptions for `batch_dot` (#10020)

* Enable Xception to work on Theano and CNTK (#10024)

* Enable Xception to work on Theano and CNTK

* Fix different predictions over all the backends

* Add support for passthrough arguments to NumpyArrayIterator (#10035)

* Add support for second output to NumpyArrayIterator

* Enable Model subclassing API (#10046)

* Refactor topological part of Keras engine.

* Fix imports

* Fix merge mixup.

* Refactor training part of the Keras engine.

* Fix unit tests.

* Refactor Network to prepare support for Model subclassing

* Finish enabling Model subclassing.

* Add `long_description` field in setup.py.

* Remove unused import

* Increase pytest duration from 10 mins to 20 mins (#10072)

* [RELNOTES] Simplify implementation of Sequential and remove legacy Merge support (#10077)

* Refactor topological part of Keras engine.

* Fix imports

* Fix merge mixup.

* Refactor training part of the Keras engine.

* Fix unit tests.

* Refactor Network to prepare support for Model subclassing

* Finish enabling Model subclassing.

* Simplify Sequential implementation.

RELNOTES: This breaks weight loading and model loading/saving for models from Keras 0.* that used Merge layers. The Merge layer has been deprecated for 2 years.

* [RELNOTES] Remove support for Keras 0.* Merge layer and associated functionality, which was scheduled for 08/2017.

* fix typo (#10078)

* Add documentation to several activation functions (#10066)

* Add documentation to several activation functions

* Fix style issues.

* Small style fixes.

* [RELNOTES] Introduce `preprocessing.image.save_img` and remove deprecated imsave method in neural style transfer example (#9996)

* imsave method in scipy.misc package is deprecated - now using imageio

* updated save_img method to use array_to_img method.  also updated the neural style transfer example to use the new save_img method

* forgot to commit changes - updated save_img

* added file_format and **kwargs parameter to save_img and updated docstring

* removed space at the end of a line in save_img method.  updated instances of imsave in example scripts with save_img method.

* added kwargs to docstring.  removed additional whitespace lines

* removed trailing whitespace

* Extensive style fixes in `preprocessing.image`.

* [RELNOTES] Allow loading external backends (#10034)

* load external backend

* safety++

* Excpetion -> ValueError

* Revert TF version to 1.7 on Travis CI. (#10101)

* New Callback: EarlyBaselineStopping (#10061)

* Initial Commit

* Continued changes

* Alpha version

* Added support to make sure that previous epochs, which may pass the
baseline test, are kept in variable history.

* Code formatting

* Code formatting

* Code formatting to address CI errors

* Code formatting for CI errors

* Initial unit test code

* Adjust for epoch being zero-based

* Code formatting

* Unit tests added

* Code formatting

* Code formatting

* Factorized the unit test

* Code formatting

* Refactored to be part of EarlyStopping and modified unit tests

* Code formatting

* Adds MobiletNetV2 to applications (#10047)

* [RELNOTES] Allow symbolic tensors to be fed to models (with TF backend) (#10087)

* Allow symbolic tensors to be fed to models (with TF backend)

* Skip tensor feeding tests when TF>=1.8 not available.

* [RELNOTES] Fix EarlyStopping API

* Fix shape mismatch in `rnn()` of tensorflow_backend (#10038)

* Fix shape mismatch in `rnn()` of tensorflow_backend

Fix the shape mismatch in `rnn()` with `not unroll` and `mask is not None` of the Tensorflow backend.

Problem: If the rnn cell has any (recurrent) `states[i]` whose shape is different from that of the `output` (`states[0]`), there will raise an `ValueError` when updating that state.

Reason: This is because the `tiled_mask_t` is not updated correctly for each `states[i]` but instead is simply copied from that of the `output` (`states[0]`).

Solution: Tile the `mask_t` with the correct shape of each `states[i]`. Notice that in a similar situation with `unroll is True`, the `tiled_mask_t` is handled correctly.

* add unit_test for rnn() with states whose shape is different from that of the output.

* Revert "add unit_test for rnn() with states whose shape is different from that of the output."

This reverts commit f1df2a58ff635bbf698444e3d7403785a92dfed1.

* Simplify the unit_test for rnn with additional states

* [RELNOTES] Default `Flatten` layer’s `data_format` argument to `None`.

* #10080 Convert CuDNN weights in nested Model. (#10081)

* #10080 Convert CuDNN weights in nested Model.

- similar problem to nesting in Bidirectional (#8860)
- quick fix: just copy/paste. I'll refactor it later
- also convert biases from H5 Dataset to np.array for reshape() (#9662)

* Refactor keras.engine.saving.preprocess_weights_for_loading().

- less duplication
- extracted conversion methods for Bidirectional/Model/Sequential

* Format docstrings to follow the project code style.

* Move tests of CuDNN RNN weight conversion to a more proper place.

- move from cudnn_recurrent_test.py to test_model_saving.py
- adjust imports to be consistent

* Make better tests for CuDNN RNN weight conversion.

- add nested models (#10080)
- various model types: add Model, Sequential (not only Sequential)
- convert weights on model, not layer (not test nesting in models)
  - test just by save/load model weights instead of calling preprocess_weights_for_loading() directly

* Check GPU support via tf in pytest skipif with short-circuit evaluation.

It seems that multiple skipif decorators get evaluated eagerly, not lazily.
Thus it fails on theano and cntk backends.

I don't understand why the tests didn't fail until now.
Maybe some change in pytest?

* Refactor (deduplicate) skipif decorators for TensorFlow and GPU.

At least multiple occurrences within a test module.
Now we can't import from `tests/` and `keras.utils.test_utils` don't se pytest.
Otherwise we can define the marker only once at all.

* Fix PEP8 (whitespace).

* Add `separable_conv1d` (#10125)

* Add `separable_conv1d` for Theano

* Add `separable_conv1d` for CNTK

* Fix preprocess_input not working with int arrays (#10134)

* Make MobileNet tests random (#10132)

* Sample weighted ImageDataGenerator (#10092)

* Add support for sample_weight in ImageDataGenerator.flow

* Added test for sample weighted imagedatagen

* clarified docs + PEP8

* pep8 - blank line

* sample_weight argument after shuffle

* RNN docstring fixes; pass `maximum_iteration` argument to `while_loop` in TF.

* Use `embedding_lookup` for `gather` in TF backend (enables partitioned variables).

* Add `data_format` argument to Conv1D; docstring style fixes.

* Further fixes.

* Fix initializers (#9963)

* fix VarianceScaling

* Correct standard initializer testing

* Add test for truncated normal in backend

* Use correct theano function for truncated_normal

* Fix sloppiness in correction

* fix PEP

* improve style initialiser fix

* fix line length

* Use NCW/NWC for conv1d data format in TF backend.

* Model loading: do not return inside `try` block.

* Rely on `Flatten` layer to do data_format conversion in ResNet50, VGG16, VGG19.

* Remove potentially misleading warning messages in `applications`.

* Fix typo.

* Remove useless super delegation.

* Remove batch_size when doing stepwise fit (#10144)

`batch_size` is always `None` , because `steps_per_epoch` and `batch_size` are mutually exclusive.
Passing batch_size seems irrelevant as `None` is its default value in `test_loop`

* Add exceptions for `fit_loop` (#10145)

* Style fixes (#10148)

* Fix ConvLSTM2D unit_forget_bias initializer

* Avoid a warning by using  'x = x * y' instead of 'x *= y' (#10158)

* Fix undefined behaviour: preprocess_input copying/not copying the input arrays (#10153)

* Add copy option for image preprocessing

* Fix unnecessary import

* Fix style

* fix test error

* Make modifications in-place instead

* vae examples fixes (#10062)

* vae examples fixes

* white spaces around op

* comments and docstring fixes

* comments and docstring fixes

* comments and docstring fixes

* fixes on docs

* docs and spacing

* Merge 2 functions together in applications MobileNetV2 (#10163)

* merge 2 functions together

_inverted_res_block() and _first_inverted_res_block() are nearly the same. Merged them into one function
1. skip "Expand" part for 0 block
2. made layers names similar to original TF graph names

* added "mobl_" to prefix, pep8 fix

* remove dropout parameter

* concise prefix name

* Remove named argument from schedule function (#10178)

* Add missing doc (#10188)

* Best to set self.built=True at the end of build() (#10191)

When using Keras on TensorFlow (in graph mode), via `tf.keras`, then calling `self.add_weight()` fails if `self.built==True`, so it is best to encourage users to set `self.built=True` at the *end* of their `build()` method, rather than just "somewhere".

* Clean up preprocessing of `depthwise_kernel` for Theano (#10189)

* Allow dynamic backends in _need_convert_kernel (#10111)

* Fix _need_convert_kernel for external backends

* Don't assume external backend won't support NASNet

* Update from review comments

* fix TensorBoard callback with unit test (#10173)

* Allow project specific config files in Keras (#10183)

* Allow project specific config files in Keras

* Clearer code, updated comment

* Update cifar10_resnet.py (#10199)

change the comments on line 133.

* Adds height_shift_range to preprocessing.image doc and adds support for indentation in auto-generated doc (#10194)

* Update README.me

* Mobilenetv2 explanation  (#10174)

* i add mobilenetv2 to the table

* dd explanation for mobilenetv2

* add explanation for mibilenetv2

* Make multi_gpu_model serializable.

* Skip tests for multi gpu

* Fix slice namespace collision in backend.

* Skip multi_gpu tests for other backends than TF

* Replace np.ceil() with a faster operation (#10184)

* Replace ceil() with faster a operation

Replacing np.ceil() with faster operations, as suggested in #8488

* Revert "Replace ceil() with faster a operation"

This reverts commit 74fe60b445377b4f1be1d0a36b4777d7ed4f1c1b.

* Replace np.ceil() with a faster operation

* Fix syntax error

* Remove trailing whitespace

* Explicitly mention using a Sequence object in val (#10223)

* Grammatical error - "inputs channels" rather than "input". (#10217)

* Fix duplicated argname: num_gpus === parts (#10228)

Signed-off-by: CUI Wei <ghostplant@qq.com>

* Fix NASNet (#10209)

* Fix NASNet

* Update weight files

* In-place split to avoid inter-device duplication (#10230)

New Benchmark by in-place split:

>> keras.application.Resnet50 224x224x3 (NCWH; NVidia Tesla P100 x 4)
 input_shape = 3x224x224, batch_size =  96 x 4: 392(images/sec) => 417(images/sec)
 input_shape = 3x299x299, batch_size =  64 x 4: 229(images/sec) => 244(images/sec)
 input_shape = 3x224x224, batch_size =   8 x 4: 148(images/sec) => 163(images/sec)

>> keras.application.InceptionV3 (NCWH; NVidia Tesla P100 x 4)
 input_shape = 3x224x224, batch_size = 128 x 4: 488(images/sec) => 526(images/sec)
 input_shape = 3x299x299, batch_size =  96 x 4: 270(images/sec) => 294(images/sec)
 input_shape = 3x224x224, batch_size =   8 x 4: 146(images/sec) => 158(images/sec)

Signed-off-by: CUI Wei <ghostplant@qq.com>

* Increase test coverages by adding invalid CNTK usecases (#10236)

* Remove Sequential.model deprecation warning (#10256)

* Remove Sequential.model deprecation warning

* Remove dead line of code

* Increase test coverages by factorizing CNTK pads (#10259)

* Refactor ImageDataGenerator (#10130)

* Create get_random_transform and refactor

* Fix style and add tests

* Add more tests

* Fix documentation error

* Fix documentation style issue

* add apply_affine_transform

* document transformation dictionary

* Doc style fix

* Remove deprecated model.model from engine/saving (#10275)

* Typo in docstring for softplus (#10277)

Softplus docstring missing a parenthesis.

* Make Dot documentation inline with Concatenate (#10271)

Doc expects a list containing 2 tensors.

* Fixes automatic doc generation problem with nested lists. Adds a new test (#10212)

* Fixes automatic doc generation problem with indented lists. Adds a new test

* Some style fixes on doc automatic generation files

* Fixes a bad space in convolutional_recurrent.py

* Changes the test_doc_auto_generation in order to include a doc string taken from the codebase. Allows text lines following nested lists

* Use count_params function for non_trainable_count. (#10280)

* load_weights will fail if shape mismatch (#10266)

Fix for #10265

* Adds to and alphabetizes documentation of Layer base class. (#10282)

* Alphabetizes and adds to layers doc.

* Responding to @cais comments

* fix spacing.  Remove in(out)bound_nodes

* Non training Batch Norm operator has bad performance for it running into tensorflow's non fused batch norm API (#10207)

* When use tensorflow as backend, let batch norm run into fused batch norm as much as possible, which has better performance.

fix issue: http://github.com/keras-team/keras/issues/10058

* In Tensorflow backend, let batch norm call to FusedBatchNorm only NHWC format, also gamma and beta are not None.

Test result:
test env: with Tensorflow(commit a543d9471047ca3f6881c87105fcbe2cdff9207d Date:   Thu May 10 17:43:30 2018, local build), python3.4, centos7.4
test cases:
  "pytest  ./tests/keras/layers/normalization_test.py"  <all passed>
  "pytest  ./tests"      <keep same result as without this commit's modification on BN>

* fix code sytle.

* 1. Add axis parameter in backend's batch_normalization functions.
2. Refine the batch_normalization function in tensorflow backend, Let's it call to fused batch norm as much as possible.

Thanks the coments from fchollet.

* Trigger

* 1. add default value -1 for parameter axis in batch_normalization function in backend.
2. fix some code style.
Thanks the comments from fchollet.

* Handle capitalised extensions in list_pictures (#10220)

#10219

* Typo fix (#10293)

* Fix doc  (#10308)

* Fix naming convention

* Add missing doc

* Fix typo

* Improve docstrings of applications (#10310)

* Add pooling options in MobileNetV2 (#10313)

* Add pooling option

* Add pooling test

* Fix doc (#10327)

Fixed doc

* Handle `mask` in `TimeDistributed` wrapper. (#10242)

* equip TimeDistributed with mask and unspecified input length

* fix bugs in theano. add test on timedistributed + masking

* skip tests on cntk with multiple unspecified time lengths.

* move static shape inference to theano_backend, add docstring, etc.

* fix format

* Split `applications` and `preprocessing` modules. (#10339)

* Split `applications` and `preprocessing` modules.

* Fix dependencies.

* Move tests for applications (#10341)

* Improve the docstring of Conv3DTranspose (#10342)

* Add depth as third dimension in docstring of
  Conv3DTranspose in convolutional.py in keras.layers

* Reduce tests for applications (#10346)

* Reduce tests for applications

* Make selection over all models random

* Add an advanced activation layer for ReLU (#10322)

The max_value argument can not be used in a layer, except
custom layer or Lambda. Hence, similarly to LeakyReLU or
for example Softmax, this PR adds a layer for ReLU,
enabling also a capped ReLU to be used.

* FIX: Tensorboard callback only supports logging Embeddings layer weights (#7766)

* Embed layer-outputs rather than layer-weights in TensorBoard callback

* Update docstring and allow multiple inputs

* Fix tests

* Renaming

* Set learning phase

* Compute embeddings in batches

* Pass embedding data explicitly

* Actually process embeddings in batches

* Allow multiple inputs and validate input data

* Add example

* Delete utils.py

* Revert uncorrectly resolved merge-conflict

* Minor renaming

* Add comment clarifying the design choice

*  Fix HDF5Matrix issue when working in conjunction with TimeSeriesGenerator (#10334)

* Fix issue when working in conjunction with TimeSeriesGenerator

The TimeSeriesGenerator class uses xrange through six which caused an IndexError

* Add test

* Add corresponding test

* Fix for python3

* Simplified code

* Fix indent

* Fix test

* Supporting channels_first data format with crossentropy losses (#9715)

* Add error message when calling `summary` on unbuilt subclassed models.

* Prepare 2.2.0 release.

* Fix a version number (#10361)

* Update to Keras Applications 1.0.2 (fixes NASNet issue).

* Add tests for inputs set dynamically (#10367)

* CuDNN RNN layers nested in TimeDistributed are not converted when loading (#10357)

* Add a unit test for CuDNNGRU conversion with TimeDistributed.

* Extract duplicated function convert_model() to _convert_model_weights().

* #10356 Convert weights of CuDNN/plain RNN nested in TimeDistributed.

Same case as for Bidirectional, except that in TimeDistributed there's only
one nested layer instead of two.

* Style fix

* Update docs for 2.2.0.

* Add spatial dropout and 3D global pooling to docs (#10373)

* spatial dropout in docs

* 3d global pooling in docs

* Doc update (#10376)

A couple of variables are "used" in two examples without being defined. For consistency with other examples where auxiliary dimensions are defined, I think it would be better to explicitly assign them a value. I just used made up values, feel free to change to whatever makes more sense!

* Preserve input shape data when serializing deferred-build Sequential models.

* Add MXNet Backend (#59)

* Adding MXNet backend template. Adding all basic Variable and Tensor operations (#1)

* add activation functions

* add activation functions

* fix some legacy

* fix some legacy

* cross entropy

* cross entropy

* fix name scoping introduced in 2.0

* fix name scoping introduced in 2.0

* Add dropout, l2_normalization, random_normal/uniform/binomial (#2)

* remove the logic for hacking RNN

* remove the logic for hacking RNN

* add pooling with utils

* add pooling with utils

* minor

* lint and name scope fix

* fix access protected var

* fix add neighbor, removed __eq__ in KerasSymbol

* fix eval function, unittest for placeholder and variable

* add unittests

* fix bug

* fix bug

* fix

* add some temporary fixes in mxnet backend. undo change to the pytest.ini

* mxnet_backend graph fix, layer support  (#3)

* add activation functions

* fix some legacy

* cross entropy

* fix name scoping introduced in 2.0

* Add dropout, l2_normalization, random_normal/uniform/binomial (#2)

* remove the logic for hacking RNN

* add pooling with utils

* add activation functions

* fix some legacy

* cross entropy

* fix name scoping introduced in 2.0

* remove the logic for hacking RNN

* add pooling with utils

* minor

* lint and name scope fix

* fix access protected var

* fix add neighbor, removed __eq__ in KerasSymbol

* fix eval function, unittest for placeholder and variable

* add unittests

* fix bug

* fix bug

* fix

* add some temporary fixes in mxnet backend. undo change to the pytest.ini

* Keras function not working is a known issue, add skip in the test

* fix random_uniform/constant

* fix legacy randomize methods

* Fix MXNet backend operator bugs. Enabled Keras backend tests

* add bias

* Add Amazon copyrights to License (#6)

* fix

* fix

* fix backend for mlp

* fix context management, add optimizers

* minor change

* undo changes on example

* fix eval

* minor cleanup

* fix some property usage

* fixing AlphaDroupout, not finished yet

* add mx model instantiate

* modifies training model construct logic, fix some tests. fix reshape layer.

* minor fix

* fix bias_add

* more fix on Dense and bias_add

* In progress commit

* fix comment

* small fix

* remove pytest.skip in conv3d. But it failed with theano backend in my workspace though.

* Add conv2d and in_topk operator for mxnet backend (#11)

* Skip BatchDot tests for Theano backend. (#12)

* BatchDot, Basic Batchnorm, Fix BiasAdd, Fix Conv2D, CodeCleanup (#14)

* Fix Conv2d shape issues and enable Conv2D UTs

* Remove redundant mxnet only unit tests

* Adding batch_dot, remove deconv, code comments and cleanup

* Remove buggy conv1d implementation

* Fix CR comments. Fix lint check issues

* Move mxnet specific code from keras engine to mxnet_backend. (#15)

* Move MXNet optimizers from keras optimizers to mxnet backend (#16)

* Fix bug in reshape. Minor rename to avoid local conflicts

* Bug fixes and enable/skip all Keras tests for mxnet backend (#21)

* test results - 374 passed, 235 skipped in 114.44 seconds

* fix/skip keras tests - tests/integration_tests, tests/keras/applications

* fix/skip keras tests - tests/keras/engine/test_topology

* fix/skip keras tests - tests/keras/engine/test_training

* fix/skip keras tests - tests/keras/legacy/

* fix/skip keras tests - tests/keras/preprocessing

* fix/skip keras tests - tests/keras/utils/

* Fix CR comments

* Fix issues in zero_padding. Fix/Enable tests/layers/convolutional_test

* Add momentum to batchnorm. Enable/skip tests in layers/core, local, merge, noise, normalization

* Skip RNN tests in keras/tests/layers/recurrent_test, wrappers_test

* Fix bug in spatial padding, enable/skip tests in loss,optimizers,callback,loss_weighting, model_saving

* Fix mxnet backend multi-gpu training (#31)

Fixing bug for mxnet backend to use multiple gpus.

* Fix performance issue - Batchnormalization, Conv operator (#35)

* Fix default axis for batchnorm layer for channels_first data_format

* Performance improvement by avoiding kernel transpose in conv operation for channels_first format

* Fix model - architecture, weights and both, load and save. (#36)

* Prepare initial version of mxnet related documentation in keras (#38)

* Skip failing unit tests for unsupported functionality in mxnet backend

* Fix pep tests reported by CI

* Use pytest module skip, revert kernel_shape logic

* remove data_format param from bias_add API

* Allow Predict() without compile for mxnet backend and enable tests.

contributor - roywei@

* Fix bug - mxnet backend should not override keras config data_format to channels_first. Only warn of low performance

* Conv3d() operator implementation for Keras2.0 using MXNet backend (#40)

* conv3d implementation for keras2.0 as MXNet backend

* conv3d implementation/testing for keras2.0 using MXNet backend

* keeping -n option in pytest.ini file

* fixed comments given by Sandeep

* Add Conv1D support for MXNet backend (#44)

* Add Conv1D support for MXNet backend

* Fix CR comments

* Conv2d transpose (#47)

* add conv2d_transpose

* conv2d transpose for both channels, enabled test case

* add detailed comments and examples, fix style issue

* enable test case in topology

* Enable performance optimization for conv operators with MXNet backend. Make MXNet default backend with this branch (#48)

* Fix conv kernel shape bug for TF backend. (#50)

* Add support for keras multi_gpu_model() API with MXNet backend (#49)

* Add support for keras multi_gpu_model() API with MXNet backend. Autoset GPU0 context on GPU machine

* Fix typo

* Add SAME padding mode support for pooling operator. (#51)

* Add rnn() operator for MXNet backend with unrolling and masking feature (#46)

* Adding rnn() operator in Keras2.0 with MXNet as backend with unroll=True and Masking=True/False and enabled relevant testcases. Also, modified couple of operators.

* Modified comments

* Added comments to a method

* Enable categorical crossentropy testcases and made minor changes

* Modified message

* nit

* Added detail description of handling variable length input in RNN

* Skip conv2d_transpose and conv3d_transpose test-case for MXNet backend and minor changes in rnn()

* Adamax and NAdam optimizer for MXNet backend (#54)

* Add Adamax optimizer for MXNet backend

* Fix lr and adamax params

* Add Nadam optimizer for mxnet backend

* Add Conv3d transpose (#52)

* conv3d tranpose, enabled test case

* update kernel shape

* replace conv2d_transpse conv3d_transpose with convnd_transpose

* update value errors with MXNet Backend info, fix typo

* add check for conv3d transpose only supports gpu with cudnn

* update context check

* diable conv3d transpose test

* fix typo in comment

* Adding MXNet backend template. Adding all basic Variable and Tensor operations (#1)

* add activation functions

* add activation functions

* fix some legacy

* fix some legacy

* cross entropy

* cross entropy

* fix name scoping introduced in 2.0

* fix name scoping introduced in 2.0

* Add dropout, l2_normalization, random_normal/uniform/binomial (#2)

* remove the logic for hacking RNN

* remove the logic for hacking RNN

* add pooling with utils

* add pooling with utils

* minor

* lint and name scope fix

* fix access protected var

* fix add neighbor, removed __eq__ in KerasSymbol

* fix eval function, unittest for placeholder and variable

* add unittests

* fix bug

* fix bug

* fix

* add some temporary fixes in mxnet backend. undo change to the pytest.ini

* mxnet_backend graph fix, layer support  (#3)

* add activation functions

* fix some legacy

* cross entropy

* fix name scoping introduced in 2.0

* Add dropout, l2_normalization, random_normal/uniform/binomial (#2)

* remove the logic for hacking RNN

* add pooling with utils

* add activation functions

* fix some legacy

* cross entropy

* fix name scoping introduced in 2.0

* remove the logic for hacking RNN

* add pooling with utils

* minor

* lint and name scope fix

* fix access protected var

* fix add neighbor, removed __eq__ in KerasSymbol

* fix eval function, unittest for placeholder and variable

* add unittests

* fix bug

* fix bug

* fix

* add some temporary fixes in mxnet backend. undo change to the pytest.ini

* Keras function not working is a known issue, add skip in the test

* fix random_uniform/constant

* fix legacy randomize methods

* Fix MXNet backend operator bugs. Enabled Keras backend tests

* add bias

* Add Amazon copyrights to License (#6)

* fix

* fix

* fix backend for mlp

* fix context management, add optimizers

* minor change

* undo changes on example

* fix eval

* minor cleanup

* fix some property usage

* fixing AlphaDroupout, not finished yet

* add mx model instantiate

* modifies training model construct logic, fix some tests. fix reshape layer.

* minor fix

* fix bias_add

* more fix on Dense and bias_add

* In progress commit

* fix comment

* small fix

* remove pytest.skip in conv3d. But it failed with theano backend in my workspace though.

* Add conv2d and in_topk operator for mxnet backend (#11)

* Skip BatchDot tests for Theano backend. (#12)

* BatchDot, Basic Batchnorm, Fix BiasAdd, Fix Conv2D, CodeCleanup (#14)

* Fix Conv2d shape issues and enable Conv2D UTs

* Remove redundant mxnet only unit tests

* Adding batch_dot, remove deconv, code comments and cleanup

* Remove buggy conv1d implementation

* Fix CR comments. Fix lint check issues

* Move mxnet specific code from keras engine to mxnet_backend. (#15)

* Move MXNet optimizers from keras optimizers to mxnet backend (#16)

* Fix bug in reshape. Minor rename to avoid local conflicts

* Bug fixes and enable/skip all Keras tests for mxnet backend (#21)

* test results - 374 passed, 235 skipped in 114.44 seconds

* fix/skip keras tests - tests/integration_tests, tests/keras/applications

* fix/skip keras tests - tests/keras/engine/test_topology

* fix/skip keras tests - tests/keras/engine/test_training

* fix/skip keras tests - tests/keras/legacy/

* fix/skip keras tests - tests/keras/preprocessing

* fix/skip keras tests - tests/keras/utils/

* Fix CR comments

* Fix issues in zero_padding. Fix/Enable tests/layers/convolutional_test

* Add momentum to batchnorm. Enable/skip tests in layers/core, local, merge, noise, normalization

* Skip RNN tests in keras/tests/layers/recurrent_test, wrappers_test

* Fix bug in spatial padding, enable/skip tests in loss,optimizers,callback,loss_weighting, model_saving

* Fix mxnet backend multi-gpu training (#31)

Fixing bug for mxnet backend to use multiple gpus.

* Fix performance issue - Batchnormalization, Conv operator (#35)

* Fix default axis for batchnorm layer for channels_first data_format

* Performance improvement by avoiding kernel transpose in conv operation for channels_first format

* Fix model - architecture, weights and both, load and save. (#36)

* Prepare initial version of mxnet related documentation in keras (#38)

* Skip failing unit tests for unsupported functionality in mxnet backend

* Fix pep tests reported by CI

* Use pytest module skip, revert kernel_shape logic

* remove data_format param from bias_add API

* Allow Predict() without compile for mxnet backend and enable tests.

contributor - roywei@

* Fix bug - mxnet backend should not override keras config data_format to channels_first. Only warn of low performance

* Conv3d() operator implementation for Keras2.0 using MXNet backend (#40)

* conv3d implementation for keras2.0 as MXNet backend

* conv3d implementation/testing for keras2.0 using MXNet backend

* keeping -n option in pytest.ini file

* fixed comments given by Sandeep

* Add Conv1D support for MXNet backend (#44)

* Add Conv1D support for MXNet backend

* Fix CR comments

* Conv2d transpose (#47)

* add conv2d_transpose

* conv2d transpose for both channels, enabled test case

* add detailed comments and examples, fix style issue

* enable test case in topology

* Enable performance optimization for conv operators with MXNet backend. Make MXNet default backend with this branch (#48)

* Fix conv kernel shape bug for TF backend. (#50)

* Add support for keras multi_gpu_model() API with MXNet backend (#49)

* Add support for keras multi_gpu_model() API with MXNet backend. Autoset GPU0 context on GPU machine

* Fix typo

* Add SAME padding mode support for pooling operator. (#51)

* Add rnn() operator for MXNet backend with unrolling and masking feature (#46)

* Adding rnn() operator in Keras2.0 with MXNet as backend with unroll=True and Masking=True/False and enabled relevant testcases. Also, modified couple of operators.

* Modified comments

* Added comments to a method

* Enable categorical crossentropy testcases and made minor changes

* Modified message

* nit

* Added detail description of handling variable length input in RNN

* Skip conv2d_transpose and conv3d_transpose test-case for MXNet backend and minor changes in rnn()

* Adamax and NAdam optimizer for MXNet backend (#54)

* Add Adamax optimizer for MXNet backend

* Fix lr and adamax params

* Add Nadam optimizer for mxnet backend

* Add Conv3d transpose (#52)

* conv3d tranpose, enabled test case

* update kernel shape

* replace conv2d_transpse conv3d_transpose with convnd_transpose

* update value errors with MXNet Backend info, fix typo

* add check for conv3d transpose only supports gpu with cudnn

* update context check

* diable conv3d transpose test

* fix typo in comment

* Rebase to latest Keras - April 3, 2018

* Add build badges

* Fix multi_gpu API bug for CPU. Fix PEP. (#64)

* Fix multi_gpu API bug for CPU. Fix PEP.

* fix embedding layer bug (#61)

* fix embedding bug

* addressed comments, enabled more test cases

* add keras test

* reduce line length

* fix style, add blank lines

* Benchmark (#55)

* add conv2d_transpose

* conv2d transpose for both channels, enabled test case

* add detailed comments and examples, fix style issue

* add benchmark scripts for resnet and imagenet data

* combine scripts

* fix args

* fix num of gpus

* update log

* multi_gpu_model only support tf

* add benchamrk scripts for synthetic data

* update read me and scripts

* add mxnet traing result table

* update on readme

* add cifar10 dataset and enable various resnet layers

* fix compile for mxnet multiple gpu

* update callbacks

* update synthetic data script, add credits

* undo new line

* update readme, addressed pr comments

* update readme

* benchmark scripts style fix (#66)

* style fix

* remove unused import, fix line too long

* adrressed pr comments

* Added keras util API for conversion of data tensor from channels_last to channels_first using MXNet backend (#65)

* Added keras util API for conversion of data tensor from channels_last to channels_first using MXNet backend

* Modified comments

* Addressed review comments and made the API more generic accross backends

* Removed shape check

* Modified comments

* Added edge cases

* moved helper method as nested

* Added RNN benchmark scripts (#69)

* Added RNN benchmark scripts

* Fixed new line in bash script

* Removed different backend code and modified comments

* Removed spacing

* Automated the wikiText2 download script

* Added dataset_util functionality to have more flexible code

* Added minor comments

* modified minor comments

* Fixed the multi-gpu context (#68)

* Update benchmark result (#70)

* update benchmark result

* update result

* simplify folder structure

* add image result

* add note

* add note

* Add MXNet Backend (#59)

* Adding MXNet backend template. Adding all basic Variable and Tensor operations (#1)

* add activation functions

* add activation functions

* fix some legacy

* fix some legacy

* cross entropy

* cross entropy

* fix name scoping introduced in 2.0

* fix name scoping introduced in 2.0

* Add dropout, l2_normalization, random_normal/uniform/binomial (#2)

* remove the logic for hacking RNN

* remove the logic for hacking RNN

* add pooling with utils

* add pooling with utils

* minor

* lint and name scope fix

* fix access protected var

* fix add neighbor, removed __eq__ in KerasSymbol

* fix eval function, unittest for placeholder and variable

* add unittests

* fix bug

* fix bug

* fix

* add some temporary fixes in mxnet backend. undo change to the pytest.ini

* mxnet_backend graph fix, layer support  (#3)

* add activation functions

* fix some legacy

* cross entropy

* fix name scoping introduced in 2.0

* Add dropout, l2_normalization, random_normal/uniform/binomial (#2)

* remove the logic for hacking RNN

* add pooling with utils

* add activation functions

* fix some legacy

* cross entropy

* fix name scoping introduced in 2.0

* remove the logic for hacking RNN

* add pooling with utils

* minor

* lint and name scope fix

* fix access protected var

* fix add neighbor, removed __eq__ in KerasSymbol

* fix eval function, unittest for placeholder and variable

* add unittests

* fix bug

* fix bug

* fix

* add some temporary fixes in mxnet backend. undo change to the pytest.ini

* Keras function not working is a known issue, add skip in the test

* fix random_uniform/constant

* fix legacy randomize methods

* Fix MXNet backend operator bugs. Enabled Keras backend tests

* add bias

* Add Amazon copyrights to License (#6)

* fix

* fix

* fix backend for mlp

* fix context management, add optimizers

* minor change

* undo changes on example

* fix eval

* minor cleanup

* fix some property usage

* fixing AlphaDroupout, not finished yet

* add mx model instantiate

* modifies training model construct logic, fix some tests. fix reshape layer.

* minor fix

* fix bias_add

* more fix on Dense and bias_add

* In progress commit

* fix comment

* small fix

* remove pytest.skip in conv3d. But it failed with theano backend in my workspace though.

* Add conv2d and in_topk operator for mxnet backend (#11)

* Skip BatchDot tests for Theano backend. (#12)

* BatchDot, Basic Batchnorm, Fix BiasAdd, Fix Conv2D, CodeCleanup (#14)

* Fix Conv2d shape issues and enable Conv2D UTs

* Remove redundant mxnet only unit tests

* Adding batch_dot, remove deconv, code comments and cleanup

* Remove buggy conv1d implementation

* Fix CR comments. Fix lint check issues

* Move mxnet specific code from keras engine to mxnet_backend. (#15)

* Move MXNet optimizers from keras optimizers to mxnet backend (#16)

* Fix bug in reshape. Minor rename to avoid local conflicts

* Bug fixes and enable/skip all Keras tests for mxnet backend (#21)

* test results - 374 passed, 235 skipped in 114.44 seconds

* fix/skip keras tests - tests/integration_tests, tests/keras/applications

* fix/skip keras tests - tests/keras/engine/test_topology

* fix/skip keras tests - tests/keras/engine/test_training

* fix/skip keras tests - tests/keras/legacy/

* fix/skip keras tests - tests/keras/preprocessing

* fix/skip keras tests - tests/keras/utils/

* Fix CR comments

* Fix issues in zero_padding. Fix/Enable tests/layers/convolutional_test

* Add momentum to batchnorm. Enable/skip tests in layers/core, local, merge, noise, normalization

* Skip RNN tests in keras/tests/layers/recurrent_test, wrappers_test

* Fix bug in spatial padding, enable/skip tests in loss,optimizers,callback,loss_weighting, model_saving

* Fix mxnet backend multi-gpu training (#31)

Fixing bug for mxnet backend to use multiple gpus.

* Fix performance issue - Batchnormalization, Conv operator (#35)

* Fix default axis for batchnorm layer for channels_first data_format

* Performance improvement by avoiding kernel transpose in conv operation for channels_first format

* Fix model - architecture, weights and both, load and save. (#36)

* Prepare initial version of mxnet related documentation in keras (#38)

* Skip failing unit tests for unsupported functionality in mxnet backend

* Fix pep tests reported by CI

* Use pytest module skip, revert kernel_shape logic

* remove data_format param from bias_add API

* Allow Predict() without compile for mxnet backend and enable tests.

contributor - roywei@

* Fix bug - mxnet backend should not override keras config data_format to channels_first. Only warn of low performance

* Conv3d() operator implementation for Keras2.0 using MXNet backend (#40)

* conv3d implementation for keras2.0 as MXNet backend

* conv3d implementation/testing for keras2.0 using MXNet backend

* keeping -n option in pytest.ini file

* fixed comments given by Sandeep

* Add Conv1D support for MXNet backend (#44)

* Add Conv1D support for MXNet backend

* Fix CR comments

* Conv2d transpose (#47)

* add conv2d_transpose

* conv2d transpose for both channels, enabled test case

* add detailed comments and examples, fix style issue

* enable test case in topology

* Enable performance optimization for conv operators with MXNet backend. Make MXNet default backend with this branch (#48)

* Fix conv kernel shape bug for TF backend. (#50)

* Add support for keras multi_gpu_model() API with MXNet backend (#49)

* Add support for keras multi_gpu_model() API with MXNet backend. Autoset GPU0 context on GPU machine

* Fix typo

* Add SAME padding mode support for pooling operator. (#51)

* Add rnn() operator for MXNet backend with unrolling and masking feature (#46)

* Adding rnn() operator in Keras2.0 with MXNet as backend with unroll=True and Masking=True/False and enabled relevant testcases. Also, modified couple of operators.

* Modified comments

* Added comments to a method

* Enable categorical crossentropy testcases and made minor changes

* Modified message

* nit

* Added detail description of handling variable length input in RNN

* Skip conv2d_transpose and conv3d_transpose test-case for MXNet backend and minor changes in rnn()

* Adamax and NAdam optimizer for MXNet backend (#54)

* Add Adamax optimizer for MXNet backend

* Fix lr and adamax params

* Add Nadam optimizer for mxnet backend

* Add Conv3d transpose (#52)

* conv3d tranpose, enabled test case

* update kernel shape

* replace conv2d_transpse conv3d_transpose with convnd_transpose

* update value errors with MXNet Backend info, fix typo

* add check for conv3d transpose only supports gpu with cudnn

* update context check

* diable conv3d transpose test

* fix typo in comment

* Adding MXNet backend template. Adding all basic Variable and Tensor operations (#1)

* add activation functions

* add activation functions

* fix some legacy

* fix some legacy

* cross entropy

* cross entropy

* fix name scoping introduced in 2.0

* fix name scoping introduced in 2.0

* Add dropout, l2_normalization, random_normal/uniform/binomial (#2)

* remove the logic for hacking RNN

* remove the logic for hacking RNN

* add pooling with utils

* add pooling with utils

* minor

* lint and name scope fix

* fix access protected var

* fix add neighbor, removed __eq__ in KerasSymbol

* fix eval function, unittest for placeholder and variable

* add unittests

* fix bug

* fix bug

* fix

* add some temporary fixes in mxnet backend. undo change to the pytest.ini

* mxnet_backend graph fix, layer support  (#3)

* add activation functions

* fix some legacy

* cross entropy

* fix name scoping introduced in 2.0

* Add dropout, l2_normalization, random_normal/uniform/binomial (#2)

* remove the logic for hacking RNN

* add pooling with utils

* add activation functions

* fix some legacy

* cross entropy

* fix name scoping introduced in 2.0

* remove the logic for hacking RNN

* add pooling with utils

* minor

* lint and name scope fix

* fix access protected var

* fix add neighbor, removed __eq__ in KerasSymbol

* fix eval function, unittest for placeholder and variable

* add unittests

* fix bug

* fix bug

* fix

* add some temporary fixes in mxnet backend. undo change to the pytest.ini

* Keras function not working is a known issue, add skip in the test

* fix random_uniform/constant

* fix legacy randomize methods

* Fix MXNet backend operator bugs. Enabled Keras backend tests

* add bias

* Add Amazon copyrights to License (#6)

* fix

* fix

* fix backend for mlp

* fix context management, add optimizers

* minor change

* undo changes on example

* fix eval

* minor cleanup

* fix some property usage

* fixing AlphaDroupout, not finished yet

* add mx model instantiate

* modifies training model construct logic, fix some tests. fix reshape layer.

* minor fix

* fix bias_add

* more fix on Dense and bias_add

* In progress commit

* fix comment

* small fix

* remove pytest.skip in conv3d. But it failed with theano backend in my workspace though.

* Add conv2d and in_topk operator for mxnet backend (#11)

* Skip BatchDot tests for Theano backend. (#12)

* BatchDot, Basic Batchnorm, Fix BiasAdd, Fix Conv2D, CodeCleanup (#14)

* Fix Conv2d shape issues and enable Conv2D UTs

* Remove redundant mxnet only unit tests

* Adding batch_dot, remove deconv, code comments and cleanup

* Remove buggy conv1d implementation

* Fix CR comments. Fix lint check issues

* Move mxnet specific code from keras engine to mxnet_backend. (#15)

* Move MXNet optimizers from keras optimizers to mxnet backend (#16)

* Fix bug in reshape. Minor rename to avoid local conflicts

* Bug fixes and enable/skip all Keras tests for mxnet backend (#21)

* test results - 374 passed, 235 skipped in 114.44 seconds

* fix/skip keras tests - tests/integration_tests, tests/keras/applications

* fix/skip keras tests - tests/keras/engine/test_topology

* fix/skip keras tests - tests/keras/engine/test_training

* fix/skip keras tests - tests/keras/legacy/

* fix/skip keras tests - tests/keras/preprocessing

* fix/skip keras tests - tests/keras/utils/

* Fix CR comments

* Fix issues in zero_padding. Fix/Enable tests/layers/convolutional_test

* Add momentum to batchnorm. Enable/skip tests in layers/core, local, merge, noise, normalization

* Skip RNN tests in keras/tests/layers/recurrent_test, wrappers_test

* Fix bug in spatial padding, enable/skip tests in loss,optimizers,callback,loss_weighting, model_saving

* Fix mxnet backend multi-gpu training (#31)

Fixing bug for mxnet backend to use multiple gpus.

* Fix performance issue - Batchnormalization, Conv operator (#35)

* Fix default axis for batchnorm layer for channels_first data_format

* Performance improvement by avoiding kernel transpose in conv operation for channels_first format

* Fix model - architecture, weights and both, load and save. (#36)

* Prepare initial version of mxnet related documentation in keras (#38)

* Skip failing unit tests for unsupported functionality in mxnet backend

* Fix pep tests reported by CI

* Use pytest module skip, revert kernel_shape logic

* remove data_format param from bias_add API

* Allow Predict() without compile for mxnet backend and enable tests.

contributor - roywei@

* Fix bug - mxnet backend should not override keras config data_format to channels_first. Only warn of low performance

* Conv3d() operator implementation for Keras2.0 using MXNet backend (#40)

* conv3d implementation for keras2.0 as MXNet backend

* conv3d implementation/testing for keras2.0 using MXNet backend

* keeping -n option in pytest.ini file

* fixed comments given by Sandeep

* Add Conv1D support for MXNet backend (#44)

* Add Conv1D support for MXNet backend

* Fix CR comments

* Conv2d transpose (#47)

* add conv2d_transpose

* conv2d transpose for both channels, enabled test case

* add detailed comments and examples, fix style issue

* enable test case in topology

* Enable performance optimization for conv operators with MXNet backend. Make MXNet default backend with this branch (#48)

* Fix conv kernel shape bug for TF backend. (#50)

* Add support for keras multi_gpu_model() API with MXNet backend (#49)

* Add support for keras multi_gpu_model() API with MXNet backend. Autoset GPU0 context on GPU machine

* Fix typo

* Add SAME padding mode support for pooling operator. (#51)

* Add rnn() operator for MXNet backend with unrolling and masking feature (#46)

* Adding rnn() operator in Keras2.0 with MXNet as backend with unroll=True and Masking=True/False and enabled relevant testcases. Also, modified couple of operators.

* Modified comments

* Added comments to a method

* Enable categorical crossentropy testcases and made minor changes

* Modified message

* nit

* Added detail description of handling variable length input in RNN

* Skip conv2d_transpose and conv3d_transpose test-case for MXNet backend and minor changes in rnn()

* Adamax and NAdam optimizer for MXNet backend (#54)

* Add Adamax optimizer for MXNet backend

* Fix lr and adamax params

* Add Nadam optimizer for mxnet backend

* Add Conv3d transpose (#52)

* conv3d tranpose, enabled test case

* update kernel shape

* replace conv2d_transpse conv3d_transpose with convnd_transpose

* update value errors with MXNet Backend info, fix typo

* add check for conv3d transpose only supports gpu with cudnn

* update context check

* diable conv3d transpose test

* fix typo in comment

* Rebase to latest Keras - April 3, 2018

* Add build badges

* Fix multi_gpu API bug for CPU. Fix PEP. (#64)

* Fix multi_gpu API bug for CPU. Fix PEP.

* fix embedding layer bug (#61)

* fix embedding bug

* addressed comments, enabled more test cases

* add keras test

* reduce line length

* fix style, add blank lines

* Benchmark (#55)

* add conv2d_transpose

* conv2d transpose for both channels, enabled test case

* add detailed comments and examples, fix style issue

* add benchmark scripts for resnet and imagenet data

* combine scripts

* fix args

* fix num of gpus

* update log

* multi_gpu_model only support tf

* add benchamrk scripts for synthetic data

* update read me and scripts

* add mxnet traing result table

* update on readme

* add cifar10 dataset and enable various resnet layers

* fix compile for mxnet multiple gpu

* update callbacks

* update synthetic data script, add credits

* undo new line

* update readme, addressed pr comments

* update readme

* benchmark scripts style fix (#66)

* style fix

* remove unused import, fix line too long

* adrressed pr comments

* Added keras util API for conversion of data tensor from channels_last to channels_first using MXNet backend (#65)

* Added keras util API for conversion of data tensor from channels_last to channels_first using MXNet backend

* Modified comments

* Addressed review comments and made the API more generic accross backends

* Removed shape check

* Modified comments

* Added edge cases

* moved helper method as nested

* Added RNN benchmark scripts (#69)

* Added RNN benchmark scripts

* Fixed new line in bash script

* Removed different backend code and modified comments

* Removed spacing

* Automated the wikiText2 download script

* Added dataset_util functionality to have more flexible code

* Added minor comments

* modified minor comments

* Fixed the multi-gpu context (#68)

* Update benchmark result (#70)

* update benchmark result

* update result

* simplify folder structure

* add image result

* add note

* add note

* rebase to latest Keras - April 20, 2018, fix bug and unit tests

* Added detailed RNN results (#73)

* Added detailed RNN results

* Modified table content and added CUDA version

* fix keras examples (#72)

* fix auto encoder examples

* update other examples

* fix style and add ctc not implemented error

* Added Detailed RNN results (#77)

* Modified RNN benchmark document

* Added minor comments

* fixed broken image link

* Added API to extract metrics from a test and also added epoch parameter (#78)

* Add mxnet backend tutorial documents (#76)

* add performance tips document

* update warning

* add docs from wiki

* add initial multi gpu doc, simplified installation doc, fix benchmark doc typo

* update install steps

* add multi_gpu_model tutorial

* Support exporting model as MXNet model (sym, params). (#80)

* Support exporting model as MXNet model (sym, params).

* Return data_names and data_shapes

* add unit tests for mxnet model save API

* Add test with LSTM layer for mxnet model save API

* Add support for functional Model graphs in save_mxnet_model API

* Add additional logging for cnn benchmarks (#89)

* add extra logging

* add logging for cnn synthetic

* fix log name

* fix file name

* Log RNN benchmark results (#90)

* Make benchmark result logging available in RNN scripts

* Make log file name consistent across CNN and RNN benchmarks

* fix pytest errors (#93)

* Cherry pick keras-team/keras 2.1.6 missing 3 commits into awslabs/keras-apache-mxnet (#96)

* update multi_gpu api in benchmark scripts (#95)

* update multi_gpu

* update logging

* fix logging

* fix logging

* fix speed format

* remove learning rate log

* Revamp keras-mxnet docs (#82)

* Update main README and move mxnet_backend_docs under docs

* revisit installation mxnet backend docs

* revisit multi_gpu_training mxnet backend docs

* revisit performance_guide mxnet backend docs

* revisit using rnn with mxnet backend in mxnet backend docs

* add save_mxnet_model tutorials in mxnet backend docs

* Fixing review comments from aaron

* Resolve CR comments on save_mxnet_model tutorial

* Fix broken links, update tutorial links in the mxnet_backend code

* revamp benchmark results readme

* Benchmark results README page revamp

* Add library versions

* Remove too detailed benchmark results. Summarize in README

* Get back detailed results document

* Remove experiemental RNN benchmarks from README

* addressed review comments on benchmark results

* Set latest stable dependency of h5py to avoid warnings

* Rebase to latest Keras April 20 2018 (#71)

* Improve tests by designating dtype of sample data (#9834)

* Document that "same" is inconsistent across backends with strides!=1 (#9629)

* Document that `"same"` is inconsistent across backends with `strides` != 1

* Use "[here](...)"

* #9642 Add kwarg and documentation for dilation_rate to SeparableConvs (#9844)

* Add kwarg and documentation for dilation_rate to SeparableConvs

* Fix pep8 complaint

I forgot to check the style before committing. Pep8 was complaining about a missing whitespace after comma, now it's fixed.

* fit/evaluate_generator supporting native tensors (#9816)

Currently, `fit/evaluate_generator` don't support this case without this fix.
But framework-native data tensors are already supported by `_fit_loop` and `_test_loop`.

Signed-off-by: CUI Wei <ghostplant@qq.com>

* Add h5py to dependencies

* Fixed typo. (#9866)

* Fix image_ocr.py example ValueError (#9869)

* Fixed the NASNet issue. (#9865)

* Fixed the NASNet issue.

* Nasnet doesn't require flatten.

* Updated documentation accordingly.

* Removed generate dropout ones from recurrent. (#9892)

* Removed generate dropout ones from recurrent.

* Fixed index issue.

* Fix `in_test_phase` of CNTK and Add its tests (#9902)

* Fix dtype designation for `variable` of CNTK and Add its tests (#9903)

* import `pydot`, improve error messages about `pydot` and GraphViz, bump to `pydot >= 1.2.4` (#9904)

* REL: bump to `pydot >= 1.2.4` in `extras_require`

* MAI: import pydot (as required in `extras_require`)

* MAI: refine error messages for `pydot` and GraphViz

distinguish between absence of `pydot` and failure to find
the executables of GraphViz in the $PATH.

* DEV: ignore `.pytest_cache`

* Fix documentation of flow_from_directory() (#9910)

The way the documentation is parsed for the Keras website made some lines of the documentation beginning with "Default:" look funny. Also changed the documentation of return value to be clear that it always returns a batch of images.

* ModelCheckpoint: print previous best (#9911)

* multi_gpu_model supporting legacy/fullCPU/fullGPU (#9638)

Signed-off-by: CUI Wei <ghostplant@qq.com>

* Fix `batch_dot` of Theano when `axes=0` (#9920)

* Fix `batch_dot` of CNTK when `axes=None` (#9921)

* Fix `batch_dot` of TensorFlow when `axes=None` (#9922)

* Fix stateful metrics when passing dict to compile (#9894)

* Added note to manually install h5py where needed (#9830)

* Added notes to manually install h5py if needed

* Added FAQ entry on h5py

* deleted redundant remark about h5py

* updated FAQ to reflect dependency change

* fixed comment format to pass failing test

* removed new trailing whitespaces

* improved docstring format

* reverted callbacks.py

* fixed links in model.py

* updated faq.py

* link pointing to FAQ

* Add support for `constants` in Bidirectional wrapper (#9260)

* Add support fot `constants` in Bidirectional wrapper

* Add more tests for Bidirectional wrapper

* Fix `compute_mask` for Birectional with return_state=True

Fix `compute_mask` to properly support `return_state` introduced in Birectional with #8977

* Add test for Bidirectional with unknown timestamps

* Skip test for CNTK for unknown timestamps with Bidirectional

* avoid override the input constant when need broadcast sequential axis on rnn's constant

* Move _standardize_args to recurrent, remove duplication

* Fix  for Birectional when multiple masks are passed

* Updated for TF 1.7 (#9937)

* fix TimeSeriesGenerator glitch (#9899)

* Added an error message for undefined shape on NASNet. (#9891)

* Added an error message for undefined shape on NASNet.

* Forgot that the message should be present only when loading imagenet weights.

* Changed the message.

* Fix PEP8

* Allow shift_range to be 1-D array-like or int (#8869)

* Allow shift_range to be 1-D array-like or int

* Add docstrings

* Fix conflict resolution merge minor disaster

* remove stray line from merge

* Remove extra "tabs"

* Exclude multi-gpu utils when reporting coverages (#9942)

* Make conv_invalid_use and pooling_invalid_use efficient (#9944)

* Chenta/cntk bn (#9952)

* fix cntk static learning phase issue; add a test

* fix code style;add more comments

* add boolean support

* fix code style issue

* Immigrate reference operations to a separate module (#9948)

* Add MXNet Backend (#59)

* Adding MXNet backend template. Adding all basic Variable and Tensor operations (#1)

* add activation functions

* add activation functions

* fix some legacy

* fix some legacy

* cross entropy

* cross entropy

* fix name scoping introduced in 2.0

* fix name scoping introduced in 2.0

* Add dropout, l2_normalization, random_normal/uniform/binomial (#2)

* remove the logic for hacking RNN

* remove the logic for hacking RNN

* add pooling with utils

* add pooling with utils

* minor

* lint and name scope fix

* fix access protected var

* fix add neighbor, removed __eq__ in KerasSymbol

* fix eval function, unittest for placeholder and variable

* add unittests

* fix bug

* fix bug

* fix

* add some temporary fixes in mxnet backend. undo change to the pytest.ini

* mxnet_backend graph fix, layer support  (#3)

* add activation functions

* fix some legacy

* cross entropy

* fix name scoping introduced in 2.0

* Add dropout, l2_normalization, random_normal/uniform/binomial (#2)

* remove the logic for hacking RNN

* add pooling with utils

* add activation functions

* fix some legacy

* cross entropy

* fix name scoping introduced in 2.0

* remove the logic for hacking RNN

* add pooling with utils

* minor

* lint and name scope fix

* fix access protected var

* fix add neighbor, removed __eq__ in KerasSymbol

* fix eval function, unittest for placeholder and variable

* add unittests

* fix bug

* fix bug

* fix

* add some temporary fixes in mxnet backend. undo change to the pytest.ini

* Keras function not working is a known issue, add skip in the test

* fix random_uniform/constant

* fix legacy randomize methods

* Fix MXNet backend operator bugs. Enabled Keras backend tests

* add bias

* Add Amazon copyrights to License (#6)

* fix

* fix

* fix backend for mlp

* fix context management, add optimizers

* minor change

* undo changes on example

* fix eval

* minor cleanup

* fix some property usage

* fixing AlphaDroupout, not finished yet

* add mx model instantiate

* modifies training model construct logic, fix some tests. fix reshape layer.

* minor fix

* fix bias_add

* more fix on Dense and bias_add

* In progress commit

* fix comment

* small fix

* remove pytest.skip in conv3d. But it failed with theano backend in my workspace though.

* Add conv2d and in_topk operator for mxnet backend (#11)

* Skip BatchDot tests for Theano backend. (#12)

* BatchDot, Basic Batchnorm, Fix BiasAdd, Fix Conv2D, CodeCleanup (#14)

* Fix Conv2d shape issues and enable Conv2D UTs

* Remove redundant mxnet only unit tests

* Adding batch_dot, remove deconv, code comments and cleanup

* Remove buggy conv1d implementation

* Fix CR comments. Fix lint check issues

* Move mxnet specific code from keras engine to mxnet_backend. (#15)

* Move MXNet optimizers from keras optimizers to mxnet backend (#16)

* Fix bug in reshape. Minor rename to avoid local conflicts

* Bug fixes and enable/skip all Keras tests for mxnet backend (#21)

* test results - 374 passed, 235 skipped in 114.44 seconds

* fix/skip keras tests - tests/integration_tests, tests/keras/applications

* fix/skip keras tests - tests/keras/engine/test_topology

* fix/skip keras tests - tests/keras/engine/test_training

* fix/skip keras tests - tests/keras/legacy/

* fix/skip keras tests - tests/keras/preprocessing

* fix/skip keras tests - tests/keras/utils/

* Fix CR comments

* Fix issues in zero_padding. Fix/Enable tests/layers/convolutional_test

* Add momentum to batchnorm. Enable/skip tests in layers/core, local, merge, noise, normalization

* Skip RNN tests in keras/tests/layers/recurrent_test, wrappers_test

* Fix bug in spatial padding, enable/skip tests in loss,optimizers,callback,loss_weighting, model_saving

* Fix mxnet backend multi-gpu training (#31)

Fixing bug for mxnet backend to use multiple gpus.

* Fix performance issue - Batchnormalization, Conv operator (#35)

* Fix default axis for batchnorm layer for channels_first data_format

* Performance improvement by avoiding kernel transpose in conv operation for channels_first format

* Fix model - architecture, weights and both, load and save. (#36)

* Prepare initial version of mxnet related documentation in keras (#38)

* Skip failing unit tests for unsupported functionality in mxnet backend

* Fix pep tests reported by CI

* Use pytest module skip, revert kernel_shape logic

* remove data_format param from bias_add API

* Allow Predict() without compile for mxnet backend and enable tests.

contributor - royw…
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

3 participants
You can’t perform that action at this time.