Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

When I use it on other models, I get the missing key error #10

Open
sunyclj opened this issue Aug 15, 2023 · 2 comments
Open

When I use it on other models, I get the missing key error #10

sunyclj opened this issue Aug 15, 2023 · 2 comments

Comments

@sunyclj
Copy link

sunyclj commented Aug 15, 2023

Missing key(s) in state_dict: "forward_resblocks.main.2.0.conv1.act_scale_pre", "forward_resblocks.main.2.0.conv1.act_scale", "forward_resblocks.main.2.0.conv2.act_scale", ....................
please,what's the reason?
Conv2D_WN_Pre contains
'self.act_scale_pre = nn.Parameter(torch.ones(in_channels).view(1, -1, 1, 1), requires_grad=True)
self.act_scale = nn.Parameter(torch.ones(out_channels).view(1, -1, 1, 1), requires_grad=True)',
does it need to be defined when training a model without pruning?

@Zj-BinXia
Copy link
Owner

需要定义,这是给后面剪枝用的,我当时做实验尝试了两点,根据act_scale的大小进行剪枝或者根据权重的L1-Norm进行剪枝,所以保留了这个设定。其实剪枝最早是在high level任务上进行的,那里的网络会使用batchNorm,这是一个天然的gate,因此有人常使用BN里面的系数大小定义一个通道的重要性进行剪枝。但是low-level任务中一般不用这个BN,所以又加了一个全部定义为1的系数来简单的衡量一下通道的重要性。

@sunyclj
Copy link
Author

sunyclj commented Aug 17, 2023

需要定义,这是给后面剪枝用的,我当时做实验尝试了两点,根据act_scale的大小进行剪枝或者根据权重的L1-Norm进行剪枝,所以保留了这个设定。其实剪枝最早是在high level任务上进行的,那里的网络会使用batchNorm,这是一个天然的gate,因此有人常使用BN里面的系数大小定义一个通道的重要性进行剪枝。但是low-level任务中一般不用这个BN,所以又加了一个全部定义为1的系数来简单的衡量一下通道的重要性。

明白了,谢谢!
请问在./basicsr/pruner/utils.py中定义了函数adjust_pr,这是什么作用呢?在这之前,已经确定了pruned_wg和kept_wg,为什么还要调整呢?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants