Feature: nadam optimizer

[cancan101]

Adam with nesterov momentum.

Keras has it: https://keras.io/optimizers/#nadam
TF implementation (old and doesn't quite work) : https://github.com/tdozat/Optimization/blob/master/tensorflow/nadam.py

[inflation]

I'm thinking about the same approach like MomentumOptimizer with a use_nesterov flag. But I don't know if it is still preferable to change the interface now.

[poxvoculi]

Is there a reason this shouldn't simply be a community supplied option, like the cited version? Are you suggesting a specific modification to TF? Otherwise perhaps it would be better to discuss such an extension on stack overflow.

[cancan101]

@poxvoculi Do you think it makes sense to add a use_nesterov flag into Adam (in the tf repos)?

[poxvoculi]

@ebrevdo: who would have an informed opinion on this?

[inflation]

@poxvoculi A C++ implementation would be faster than python hacks like above. Also, maybe an awaiting googler label?

[ebrevdo]

Paul: probably vincent would know how popular / useful this optimizer is for the general case.

…

On Wed, Feb 22, 2017 at 11:53 AM, Aaron Hu ***@***.***> wrote: @poxvoculi <https://github.com/poxvoculi> A C++ implementation would be faster than python hacks like above. Also, maybe an awaiting googler label? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#7715 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABtim1Y1n3pYI1s1-68zine8Pr6ees1Jks5rfJJHgaJpZM4MGufx> .

[vincentvanhoucke]

Plausible. I have no experience with it. If someone implements it, it sounds like something we would incorporate.

[girving]

Automatically closing due to lack of recent activity. Please update the issue when new information becomes available, and we will reopen the issue. Thanks!