ONNX export: Instance normalization, Shape #12920

vandanavk · 2018-10-23T00:49:08Z

Description

Added support for exporting Instance Normalization, Shape to ONNX

Checklist

Essentials

Please feel free to remove inapplicable items for your PR.

The PR title starts with [MXNET-$JIRA_ID], where $JIRA_ID refers to the relevant JIRA issue created (except PRs with tiny changes)
Changes are complete (i.e. I finished coding on this PR)
All changes have test coverage:
Unit tests are added for small changes to verify correctness (e.g. adding a new operator)
Nightly tests are added for complicated/long-running ones (e.g. changing distributed kvstore)
Build tests will be added for build configuration changes (e.g. adding a new build option with NCCL)
Code is well-documented:
For user-facing API changes, API doc string has been updated.
For new C++ functions in header files, their functionalities and arguments are documented.
For new examples, README.md is added to explain the what the example does, the source of the dataset, expected performance on test set and reference to the original paper if applicable
Check the API doc at http://mxnet-ci-doc.s3-accelerate.dualstack.amazonaws.com/PR-$PR_ID/$BUILD_ID/index.html
To the my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change

Changes

Added export in _op_translations.py
Added test in onnx_backend_test.py
Created a common backend_rep.py. Removed python-pytest/onnx/export/backend_rep.py and python-pytest/onnx/import/mxnet_backend_rep.py
In the PR Onnx version update from 1.2.1 to 1.3 in CI #12633, changes were made to use different bind method based on batch sizes, in ONNX import. Using sym.bind by default.

Comments

All tests in onnx_backend_test and mxnet_backend_test pass

roywei · 2018-10-23T20:01:15Z

@vandanavk Thanks for the contribution!
@mxnet-label-bot [pr-work-in-progress]

vandanavk · 2018-10-26T18:41:48Z

@sandeep-krishnamurthy Please change the label to pr-awaiting-review.

@Roshrini @anirudhacharya @zhreshold for review

zhreshold · 2018-10-26T19:58:05Z

tests/python-pytest/onnx/import/mxnet_backend_rep.py

-        data_shapes = []
-        for idx, input_name in enumerate(data_names):
-            data_shapes.append((input_name, inputs[idx].shape))
+        if self.arg_params:


is this duplicate of MXNetBackendRep in tests/python-pytest/onnx/export/backend_rep.py ?

Agree. Will create a common file in tests/python-pytest/onnx/ and make changes in other files accordingly.

vandanavk · 2018-10-29T17:23:50Z

this PR is ready for review again @zhreshold @anirudhacharya @Roshrini

zhreshold · 2018-11-01T18:32:00Z

Please rebase to the changes with #12878 merged already.

vandanavk · 2018-11-01T20:02:17Z

Rebased. Ready for review again

vandanavk · 2018-11-02T19:34:47Z

Since sym.bind doesn't have the option to allow_missing, this fails for certain models - as faced by Roshani.

In forward_pass(), the output label is removed from data_names. mod.set_params doesn't complain about this because of allow_missing=True, but in sym.bind allow_missing is always False, and hence when the output_label appears in sym.list_arguments() but not in the args passed, it throws an error.

one solution is to bring back the check in export_onnx.py to do the following:

if batch size same
      mod.bind 
else
      sym.bind

zhreshold · 2018-11-02T22:48:52Z

Since sym.bind doesn't have the option to allow_missing, this fails for certain models - as faced by Roshani.

In forward_pass(), the output label is removed from data_names. mod.set_params doesn't complain about this because of allow_missing=True, but in sym.bind allow_missing is always False, and hence when the output_label appears in sym.list_arguments() but not in the args passed, it throws an error.

one solution is to bring back the check in export_onnx.py to do the following:
if batch size same
      mod.bind 
else
      sym.bind

Try manual check

vandanavk · 2018-11-26T19:54:46Z

@mxnet-label-bot update [ONNX, pr-awaiting-review]

vandanavk · 2018-11-27T17:47:56Z

@zhreshold This PR is ready for review again. The issues previously seen do not occur now because of Sina's changes in #13390

vandanavk · 2018-11-27T22:31:39Z

@mxnet-label-bot update [ONNX, pr-awaiting-merge]

sandeep-krishnamurthy

LGTM.
Can you please rebase?

vandanavk · 2018-11-30T23:08:17Z

@marcoabreu Is this PR good to go? centos-cpu, windows-gpu are failing.

marcoabreu · 2018-12-01T00:00:02Z

Yeah please bare with me while we are transitioning to the new CI pipeline.i will send an email on dev@ as soon as I'm done.

…ile (#13478) * updated to v1.5.0 * Bumped minor version from 1.4.0 to 1.5.0 on master * added Anirudh as maintainer for R package ... adding something useful and re-trigger PR check * Updated license file for clojure, onnx-tensorrt, gtest, R-package * Get the correct include path in pip package (#13452) * add find_include_path API * address reviewer comment * change return type from list to string * add unit test * address reviewer comment * address reviewer comment * address reviewer comment * address reviewer comment * fix include path problem in pip package * add comment * fix lint error * address reviewer comment * address reviewer comment * Use ~/.ccache as default ccache directory so is not cache is not erased on reboot (#13431) * Skip flaky test #13446 (#13480) * Rewrite dataloader with process pool, improves responsiveness and reliability (#13447) * fix recordio.py * rewrite dataloader with pool * fix batch as tuple * fix prefetching * fix pylint * picklable function * use pickle * add missing commit * Fix errors in docstrings for subgraph op; use code directive (#13463) * [MXNET-1158] JVM Memory Management Documentation (#13105) * update train_mnist * Add documentation for JVM Memory Management * update doc * address nit picks * address nit picks * Grammar and clarity edits for memory management doc * Edits for scala memory management * Update memory-management.md * Update memory-management.md * Update memory-management.md * capitalization fix * Update row_sparse tutorial (#13414) Update row_sparse tutorial * Add resiliency to onnx export code (#13426) * Added resiliency to onnx export code - With previous infer-shape implementation, if input shape was list instead of tuple or if extra non-existent parameters were provided, the code would still work. The fixes in this commit make sure that behavior is restored to prevent any compatibility issues with existing export code. * Fixed name of net in unittest * Fix pylint * [MXNET-1185] Support large array in several operators (part 1) (#13418) * fix a few operators with large arrays (# of elements) * fix bug in broadcast_div and add tests * address reviewer comment * add unit test * add empty line * retrigger CI * [MXNET-1210 ] Gluon Audio - Example (#13325) * Initialized the example * Addressed PR comments, about existing synset.txt file - no overwrite * RST - docstring issues fixed * added README * Addressed PR comments * Addressed PR comments, checking Divide by 0 * Raising error if format is not supported. * changed a line for ndarray of labels * Trigger CI * Trigger CI * PR comments addressed around skip_header argument * Addressed PR comments around librosa import * PR Comments * Passing lazy=lazy from argument * Added PR comments, labels to README.MD * Trigger CI * Addressing PR Comments in README * Modified README.md * Added example under audio folder * Retrigger CI * Retrigger CI * ONNX export: Instance normalization, Shape (#12920) * ONNX import/export: Make backend_rep common * ONNX export: Instance Normalization * ONNX export: Shape operator * Clarify dependency on OpenCV in CNN Visualization tutorial. (#13495) * clarify ops faq regarding docs strings (#13492) * Add graph_compact operator. (#13436) * add graph_compact. * fix. * add doc. * add tests for graph_compact. * address comments. * update docs. * trigger CI * Deprecate Jenkinsfile (#13474) * update github location for sampled_block.py (#13508) Updated to https://github.com/dmlc/gluon-nlp/blob/master/src/gluonnlp/model/sampled_block.py * #13453 [Clojure] - Add Spec Validations to the Optimizer namespace (#13499) * ONNX export: Logical operators (#12852) * Fix cmake options parsing in dev_menu (#13458) Add GPU+MKLDNN unittests to dev_menu * Revert "Manually track num_max_thread (#12380)" (#13501) This reverts commit 7541021. * Feature/mkldnn static 2 (#13503) * build mkldnn as static lib * update makefile to statically build mkldnn * build static mkldnn * fix static name * fix static name * update static for mac * rename mkldnn dep in ci * remove moving mkldnn dynamic lib * remove commented code * remove mkldnn dnaymic for unitest * force static for mkldnn lib * remove dynamic mkldnn bind * only link windows * add mkldnn.mk * try force linking * remove mkldnn dynanmic check * remove test mkldnn install * fix spacing * fix index * add artifacts * add comment about windows * remove static * update makefile * fix toctree Sphinx errors (#13489) * fix toctree errors * nudging file for CI * Disabled flaky test test_gluon_data.test_recordimage_dataset_with_data_loader_multiworker (#13527) * [MXNET-1234] Fix shape inference problems in Activation backward (#13409) * Provide a failing test for ReLU activation shape inference bug * Fix Activation backward shape inference fixes: #13333 * Add softsign Activation to test_gluon.py * Use activation in GPU if we are using CUDNN and not MKLDNN as it's happening right now * Don't disable MKLDNN

* ONNX import/export: Make backend_rep common * ONNX export: Instance Normalization * ONNX export: Shape operator

…ile (apache#13478) * updated to v1.5.0 * Bumped minor version from 1.4.0 to 1.5.0 on master * added Anirudh as maintainer for R package ... adding something useful and re-trigger PR check * Updated license file for clojure, onnx-tensorrt, gtest, R-package * Get the correct include path in pip package (apache#13452) * add find_include_path API * address reviewer comment * change return type from list to string * add unit test * address reviewer comment * address reviewer comment * address reviewer comment * address reviewer comment * fix include path problem in pip package * add comment * fix lint error * address reviewer comment * address reviewer comment * Use ~/.ccache as default ccache directory so is not cache is not erased on reboot (apache#13431) * Skip flaky test apache#13446 (apache#13480) * Rewrite dataloader with process pool, improves responsiveness and reliability (apache#13447) * fix recordio.py * rewrite dataloader with pool * fix batch as tuple * fix prefetching * fix pylint * picklable function * use pickle * add missing commit * Fix errors in docstrings for subgraph op; use code directive (apache#13463) * [MXNET-1158] JVM Memory Management Documentation (apache#13105) * update train_mnist * Add documentation for JVM Memory Management * update doc * address nit picks * address nit picks * Grammar and clarity edits for memory management doc * Edits for scala memory management * Update memory-management.md * Update memory-management.md * Update memory-management.md * capitalization fix * Update row_sparse tutorial (apache#13414) Update row_sparse tutorial * Add resiliency to onnx export code (apache#13426) * Added resiliency to onnx export code - With previous infer-shape implementation, if input shape was list instead of tuple or if extra non-existent parameters were provided, the code would still work. The fixes in this commit make sure that behavior is restored to prevent any compatibility issues with existing export code. * Fixed name of net in unittest * Fix pylint * [MXNET-1185] Support large array in several operators (part 1) (apache#13418) * fix a few operators with large arrays (# of elements) * fix bug in broadcast_div and add tests * address reviewer comment * add unit test * add empty line * retrigger CI * [MXNET-1210 ] Gluon Audio - Example (apache#13325) * Initialized the example * Addressed PR comments, about existing synset.txt file - no overwrite * RST - docstring issues fixed * added README * Addressed PR comments * Addressed PR comments, checking Divide by 0 * Raising error if format is not supported. * changed a line for ndarray of labels * Trigger CI * Trigger CI * PR comments addressed around skip_header argument * Addressed PR comments around librosa import * PR Comments * Passing lazy=lazy from argument * Added PR comments, labels to README.MD * Trigger CI * Addressing PR Comments in README * Modified README.md * Added example under audio folder * Retrigger CI * Retrigger CI * ONNX export: Instance normalization, Shape (apache#12920) * ONNX import/export: Make backend_rep common * ONNX export: Instance Normalization * ONNX export: Shape operator * Clarify dependency on OpenCV in CNN Visualization tutorial. (apache#13495) * clarify ops faq regarding docs strings (apache#13492) * Add graph_compact operator. (apache#13436) * add graph_compact. * fix. * add doc. * add tests for graph_compact. * address comments. * update docs. * trigger CI * Deprecate Jenkinsfile (apache#13474) * update github location for sampled_block.py (apache#13508) Updated to https://github.com/dmlc/gluon-nlp/blob/master/src/gluonnlp/model/sampled_block.py * apache#13453 [Clojure] - Add Spec Validations to the Optimizer namespace (apache#13499) * ONNX export: Logical operators (apache#12852) * Fix cmake options parsing in dev_menu (apache#13458) Add GPU+MKLDNN unittests to dev_menu * Revert "Manually track num_max_thread (apache#12380)" (apache#13501) This reverts commit 7541021. * Feature/mkldnn static 2 (apache#13503) * build mkldnn as static lib * update makefile to statically build mkldnn * build static mkldnn * fix static name * fix static name * update static for mac * rename mkldnn dep in ci * remove moving mkldnn dynamic lib * remove commented code * remove mkldnn dnaymic for unitest * force static for mkldnn lib * remove dynamic mkldnn bind * only link windows * add mkldnn.mk * try force linking * remove mkldnn dynanmic check * remove test mkldnn install * fix spacing * fix index * add artifacts * add comment about windows * remove static * update makefile * fix toctree Sphinx errors (apache#13489) * fix toctree errors * nudging file for CI * Disabled flaky test test_gluon_data.test_recordimage_dataset_with_data_loader_multiworker (apache#13527) * [MXNET-1234] Fix shape inference problems in Activation backward (apache#13409) * Provide a failing test for ReLU activation shape inference bug * Fix Activation backward shape inference fixes: apache#13333 * Add softsign Activation to test_gluon.py * Use activation in GPU if we are using CUDNN and not MKLDNN as it's happening right now * Don't disable MKLDNN

vandanavk requested a review from szha as a code owner October 23, 2018 00:49

vandanavk mentioned this pull request Oct 23, 2018

[MXNET-886] ONNX export: HardSigmoid, Less, Greater, Equal #12812

Merged

5 tasks

marcoabreu added the pr-work-in-progress PR is still work in progress label Oct 23, 2018

vandanavk force-pushed the onnx_bugs branch from 5176bc1 to 0394bc3 Compare October 25, 2018 21:36

vandanavk changed the title ~~[WIP] ONNX export: Instance normalization~~ [WIP] ONNX export: Instance normalization, Shape Oct 25, 2018

vandanavk force-pushed the onnx_bugs branch 3 times, most recently from 64f20b3 to 610a5b0 Compare October 26, 2018 17:53

vandanavk changed the title ~~[WIP] ONNX export: Instance normalization, Shape~~ ONNX export: Instance normalization, Shape Oct 26, 2018

vandanavk force-pushed the onnx_bugs branch from 610a5b0 to c206d69 Compare October 26, 2018 18:23

zhreshold suggested changes Oct 26, 2018

View reviewed changes

vandanavk force-pushed the onnx_bugs branch 3 times, most recently from 2d19b15 to 00d1e0c Compare October 28, 2018 04:06

vandanavk force-pushed the onnx_bugs branch 2 times, most recently from da6a48a to a2e0cb2 Compare November 1, 2018 20:00

vandanavk force-pushed the onnx_bugs branch 2 times, most recently from acf64f2 to 4ecef7e Compare November 26, 2018 17:41

marcoabreu added ONNX pr-awaiting-review PR is waiting for code review and removed pr-work-in-progress PR is still work in progress labels Nov 26, 2018

zhreshold approved these changes Nov 27, 2018

View reviewed changes

marcoabreu added pr-awaiting-merge Review and CI is complete. Ready to Merge and removed pr-awaiting-review PR is waiting for code review labels Nov 27, 2018

sandeep-krishnamurthy approved these changes Nov 29, 2018

View reviewed changes

vandanavk added 3 commits November 28, 2018 22:09

ONNX import/export: Make backend_rep common

6221de3

ONNX export: Instance Normalization

4884b2c

ONNX export: Shape operator

d2c6986

vandanavk force-pushed the onnx_bugs branch from 4ecef7e to d2c6986 Compare November 29, 2018 06:15

sandeep-krishnamurthy merged commit 77510d7 into apache:master Dec 1, 2018

zhaoyao73 pushed a commit to zhaoyao73/incubator-mxnet that referenced this pull request Dec 13, 2018

ONNX export: Instance normalization, Shape (apache#12920)

58ddc48

* ONNX import/export: Make backend_rep common * ONNX export: Instance Normalization * ONNX export: Shape operator

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ONNX export: Instance normalization, Shape #12920

ONNX export: Instance normalization, Shape #12920

vandanavk commented Oct 23, 2018 •

edited

roywei commented Oct 23, 2018

vandanavk commented Oct 26, 2018

zhreshold Oct 26, 2018

vandanavk Oct 26, 2018

vandanavk commented Oct 29, 2018

zhreshold commented Nov 1, 2018

vandanavk commented Nov 1, 2018

vandanavk commented Nov 2, 2018

zhreshold commented Nov 2, 2018

vandanavk commented Nov 26, 2018

vandanavk commented Nov 27, 2018

vandanavk commented Nov 27, 2018

sandeep-krishnamurthy left a comment

vandanavk commented Nov 30, 2018

marcoabreu commented Dec 1, 2018

ONNX export: Instance normalization, Shape #12920

ONNX export: Instance normalization, Shape #12920

Conversation

vandanavk commented Oct 23, 2018 • edited

Description

Checklist

Essentials

Changes

Comments

roywei commented Oct 23, 2018

vandanavk commented Oct 26, 2018

zhreshold Oct 26, 2018

Choose a reason for hiding this comment

vandanavk Oct 26, 2018

Choose a reason for hiding this comment

vandanavk commented Oct 29, 2018

zhreshold commented Nov 1, 2018

vandanavk commented Nov 1, 2018

vandanavk commented Nov 2, 2018

zhreshold commented Nov 2, 2018

vandanavk commented Nov 26, 2018

vandanavk commented Nov 27, 2018

vandanavk commented Nov 27, 2018

sandeep-krishnamurthy left a comment

Choose a reason for hiding this comment

vandanavk commented Nov 30, 2018

marcoabreu commented Dec 1, 2018

vandanavk commented Oct 23, 2018 •

edited