-
Notifications
You must be signed in to change notification settings - Fork 7
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
你好 我在渐进式剪枝这一环节有些疑问 希望您解答 #30
Comments
你好,
|
所以我可以理解为应该是M=0代表越重要所以需要保留吗?也就是楼主实际上说错了,楼主以为重要性小才会趋于0.传统掩码剪枝一般是为1保留为0修剪。但是您的工作中是把Mask矩阵看作修剪矩阵,所以1代表修剪0代表不剪,也就是1代表权重不重要0代表重要 |
你好,很出色的工作!
我在渐进式剪枝这一部分有一些疑问,在论文中,您提出用累计梯度的大小作为alpha参数重要性的衡量因子,不过在论文中好像并未说明更新alpha时变动的元素是alpha的grad值大还是值小的元素?
在代码中,我看到:
这一部分似乎说明,在update过程中是grad比threshold大的部分参数得到了更新,将其趋于0,这里我有些疑问,为何grad比threshold大的部分参数重要性就小以至于能趋于0呢?希望您解答
The text was updated successfully, but these errors were encountered: