Add NovoGrad optimizer

[DT6A]

NovoGrad optimizer was presented in this paper: https://arxiv.org/pdf/1905.11286.pdf
It was used to train the ASR model named Jasper: https://arxiv.org/pdf/1904.03288.pdf

[google-cla]

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

[mtthss]

There are a couple of conflicts, could you sync and merge so we get this submitted?

[DT6A]

There are a couple of conflicts, could you sync and merge so we get this submitted?

Hello! Yes, I will try to fix as soon as possible

[hbq1]

Could you also add novograd to the Common Optimizers section?

[hbq1]

"A GradientTransformation object."

[hbq1]

Thanks a lot for the contribution!

[hbq1]

@DT6A could you please add a test for the new optimizer?

[DT6A]

@hbq1 is there any reference how the test should be done?

[mkunesch]

Hi! We are in the process of writing instructions on this at the moment - in your case it should be enough to just add the new optimizer to the list of optimizers under test in alias_test.py here. Let me know if you have any questions on this!

[DT6A]

@mkunesch Thanks, added it

[mkunesch]

Thank you!
(If there are details of the optimizer that would be important to test in isolation it would be good to add a unit test for them, but from a quick look at the documentation and code nothing stood out to me)

[hbq1]

Merged! Thanks again for this great contribution 🎉