New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
6层的roberta模型啥时候发布呀? #22
Comments
争取本周内 |
这周能发布么? 挺着急想试试这小模型的效果 |
延期了 |
请问目前有确定6层模型以及训练语料的发放日期么?想试一试小模型的效果,谢谢🙏 |
有的,6层的在训练,就这两天就会发布。 |
太好了!albert我昨天才看到,似乎用了大量TPU训练,如果能够发布真的是太好了,非常感谢~ |
@csy1998 @Jethu1 超小模型,参数量和模型大小为bert的十分之一,训练速度加快了1倍,可以试用了 |
hi,请问楼主,6层的roberta模型,大约能在什么时候发布呢? |
roberta 6层模型在我这边的文本相似度匹配数据集上准确率提升2%,albert 6层模型提升有1%, albert 4层模型不仅不能提升任务效果,反而降低0.5%左右。但是从cpu的预测速度来看,4层以下的bert模型才能满足cpu响应性能需求,英文担心蒸馏后损失模型效果,希望作者和大家一起探究一下roberta 3-4层的模型的效果和性能如何。 |
No description provided.
The text was updated successfully, but these errors were encountered: