The purpose of this work, set as a piece of coursework, was to implement the WAME optimiser [1] and use it to train a CNN on a dataset of our choosing. We implemented the optimiser in Tensorflow 2.0 and built a model to predict the classes in the EMNIST [2] dataset. Overall we achieve a 94.6% accuracy, increasing to 99.1% for top-2 accuracy.
[1] A. Mosca and G. D. Magoulas, “Training Convolutional Networks with Weightwise Adaptive Learning Rates,” in ESANN 2017, 2017. [Online]. Available: https://www.elen.ucl.ac.be/Proceedings/esann/esannpdf/es2017-50.pdf
[2] G. Cohen, S. Afshar, J. Tapson, and A. van Schaik, “EMNIST: an extension of MNIST to handwritten letters,” feb 2017. [Online]. Available: http://arxiv.org/abs/1702.05373