Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

3.3.3小节学习速率衰减与AdamOptimizer并无因果关系 #17

Open
lilbedwin opened this issue Apr 10, 2019 · 0 comments
Open

3.3.3小节学习速率衰减与AdamOptimizer并无因果关系 #17

lilbedwin opened this issue Apr 10, 2019 · 0 comments

Comments

@lilbedwin
Copy link

文中描述的学习速率衰减是将lr用placeholder的方式,feed进优化过程。在不同的train step间,通过python进行衰减计算。
任何optimizer理论上都可以采用这种技术,文中的如下描述:

“可以采用更好的优化算法,例如 AdamOptimizer。随着迭代过程的次数,学习速率将指 数级衰减,在模型训练后期可以得到一个更稳定的精度和损失曲线。”

或许会给人造成误解,认为lr衰减是某些特定optimizer(例如AdamOptimizer)引入的特性。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant