We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
看起来参数是随机丢弃了, 有没有对比实验,不丢弃参数,但是只scale参数了
The text was updated successfully, but these errors were encountered:
这个实验我们没有尝试过,但模型的表现很大概率上会变差。
从模型的计算角度讲,不丢弃参数只rescale参数会导致模型的输出期望发生变化(相当于整体放大或者缩小了模型的输出),很有可能导致较差的结果。我们的rescale操作是为了保证在drop一部分参数的后,模型的输出期望大致不变。
Sorry, something went wrong.
我先关闭这个issue啦。
如果后续有问题可以随时重启这个issue。
No branches or pull requests
看起来参数是随机丢弃了, 有没有对比实验,不丢弃参数,但是只scale参数了
The text was updated successfully, but these errors were encountered: