Feature request: Include the Adam algorithm

[renatobellotti]

Hi,

I would like to suggest including the Adam algorithm into Optim.jl. Adam is used extensively in the deep learning community, where it is the go-to algorithm. I found a small implementation: https://gist.github.com/vankesteren/96207abcd16ecd01a2491bcbec12c73f

[pkofod]

I have Adam and AdaMax here https://github.com/JuliaNLSolvers/NLSolvers.jl

[renatobellotti]

Nice, thank you! Is it already compatible with the Optim.jl API? I couldn't find it in the documentation of Optim.jl... Would be great to have more options to experiment with. :)

[pkofod]

Sorry. I never had a need for this, but since it was requested again, I made a PR here though it may not longer be relevant for you :) #1069