ADD CapsNet example

[Soonhwan-Kwon]

Description

This example is MXNet implementation of CapsNet:
Sara Sabour, Nicholas Frosst, Geoffrey E Hinton. Dynamic Routing Between Capsules. NIPS 2017
The current best test error is 0.5%

Checklist

Essentials

• Passed code style checking (make lint)
• Changes are complete (i.e. I finished coding on this PR)
• All changes have test coverage
• For user-facing API changes, API doc string has been updated. For new C++ functions in header files, their functionalities and arguments are well-documented.
• To my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change

Changes

• ADD CapsNet example

[Soonhwan-Kwon]

We're planning to make this capsule net example as gluon also. Is there anything to do please let me know.

[piiswrong]

what does this mean?

[Soonhwan-Kwon]

If you mean the last line, Pull Request can takes days to accept, meanwhile some follow-up changes can be updated to our temporary repository https://github.com/samsungsds-rnd/capsnet.mxnet.

[piiswrong]

Does the results match the original paper?

[Soonhwan-Kwon]

The original paper's result was 0.25% and we expect close accuracy(0.34%) can be achieved by adding data augmentation(scale shift) like other implementations of other platforms(keras, tensorflow) which achieved 0.34 % (the average) result. The result of 0.25 is hard to achieve but we expect close enough accuracy can be obtained soon.

[Soonhwan-Kwon]

I expect that there are many users of MXNet finding implementation of CapsNet. We're in the middle of implementing data augmentation and will make effort to follow up the accuracy soon.

[Soonhwan-Kwon]

@piiswrong We added data augmentation(shifted up to 2 pixels) as on paper, and fixed structure by blocking gradient on both bias(as constant) and inputs_hat before routing ends. Now it achieves test error =0.44% at epoch 40.

[Soonhwan-Kwon]

@piiswrong
We updated test error=0.41% at epoch 62 by parameter tuning, and updated readme and add option for decaying factor to ease parameter tuning. The test error 0.41% and training time is around 36 sec/epoch, and it shows fastest train time among other implementations(2017-11-21).

Implementation	test err(%)	※train time/epoch	GPU Used
MXNet	0.41	36 sec	2 GTX 1080
tensorflow	0.49	※ 10 min	Unknown(4GB Memory)
Keras	0.30	55 sec	2 GTX 1080 Ti

※ tensorflow implementation's batch size is not 100 but 128 but MXNet and Keras implementation 's batchsize are 100.

Sorry for repeating update but would you mind noticing us that it can be merged or not? I'm afraid that there is no response since 2 days ago, and it seems that it is not acceptable as example for MXNet.