-
Notifications
You must be signed in to change notification settings - Fork 154
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
nesterov momentum is wrong in sgd #27
Comments
Yeah I think the nesterov mode in But maybe I'm misreading the nesterov option in |
If the learning rate is a constant, the implemented nesterov mode should behave like NAG. |
I suggest we keep this one as it is. I think it is not wrong, but an approximation. The gist @clementfarabet posted is another implementation which I will put in a separate file. |
Thanks for the clarification Ivo. Once Koray adds it in we should probably compare them. Clément
|
I just added it in. |
Cool, thanks Koray! |
@clementfarabet elaborate here?
The text was updated successfully, but these errors were encountered: