Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable gradient clipping. #1894

Closed
qingqing01 opened this issue Apr 25, 2017 · 2 comments · Fixed by #2216
Closed

Enable gradient clipping. #1894

qingqing01 opened this issue Apr 25, 2017 · 2 comments · Fixed by #2216

Comments

@qingqing01
Copy link
Contributor

This issue is related to #775 and #1891. I'll try to fix it. On the other hand, is it necessary to enable gradient and error clipping in gru-memory and lstm-memory which are implemented in one cpp file (not in the recurrent layer group)? @hedaoyuan @lcy-seso

@qingqing01 qingqing01 self-assigned this Apr 25, 2017
@lcy-seso
Copy link
Contributor

lcy-seso commented May 3, 2017

Is it necessary to implement norm gradient clipping in PaddlePaddle, for example, constraint a certain norm of gradient matrix by a threshold?

@lcy-seso
Copy link
Contributor

lcy-seso commented May 3, 2017

Gradient clipping is critical to a stable training process of RNN models, this issue #1961 is also related to gradient clipping.

@luotao1 luotao1 added this to 已有BUG in V2 API Enhancement May 9, 2017
@luotao1 luotao1 added the Bug label May 9, 2017
@lcy-seso lcy-seso added this to Top priorities in Defects board May 10, 2017
@lcy-seso lcy-seso moved this from Not in schedule to Next Week in Defects board May 10, 2017
@lcy-seso lcy-seso moved this from Next Week to Not in schedule in Defects board May 10, 2017
@lcy-seso lcy-seso moved this from Not in schedule to Done in Defects board May 22, 2017
@luotao1 luotao1 moved this from 全局BUG to 已完成 in V2 API Enhancement May 25, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Development

Successfully merging a pull request may close this issue.

3 participants