Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

initial value used in rmsprop #30

Open
yorkerlin opened this Issue Aug 23, 2015 · 2 comments

Comments

Projects
None yet
2 participants
@yorkerlin
Copy link

yorkerlin commented Aug 23, 2015

@bayerj
The 152-th line of rmsprop.py

        self.moving_mean_squared = 1

I think the initial value should be 0 instead of 1.
Any reason why 1 is better than 0

@yorkerlin

This comment has been minimized.

Copy link
Author

yorkerlin commented Aug 23, 2015

@bayerj

This comment has been minimized.

Copy link
Contributor

bayerj commented Sep 21, 2015

The reason is that it starts out as gradient descent then and transitions to rmsprop then. I have no idea if this change has a huge impact. If someone shows that it works better, I am happy with changing it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.