We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
我发现在训练过程中,总是存在一些数据会在迭代过程中发散。并且,在训练过程中,loss总是根据数据的不同,来回震荡。请教下作者针对这两个问题,有什么解决办法呢?
The text was updated successfully, but these errors were encountered:
你好,请问你解决这个问题了吗?
Sorry, something went wrong.
在经过初始化训练后,可以通过合理设置数据混合方式来解决大部分发散问题。通过约束异常loss的反向传播,可以使训练持续训练下去。loss仍然是来回震荡的。但可以在验证集上观察到下降趋势,并根据此选取合适的模型。
你好,请问你解决这个问题了吗? 在经过初始化训练后,可以通过合理设置数据混合方式来解决大部分发散问题。通过约束异常loss的反向传播,可以使训练持续训练下去。loss仍然是来回震荡的。但可以在验证集上观察到下降趋势,并根据此选取合适的模型。
好的,谢谢您的回复
你好,能把训练代码给参考下吗
No branches or pull requests
我发现在训练过程中,总是存在一些数据会在迭代过程中发散。并且,在训练过程中,loss总是根据数据的不同,来回震荡。请教下作者针对这两个问题,有什么解决办法呢?
The text was updated successfully, but these errors were encountered: