Hi, I've been using learn2learn 0.1.2 version. I was wondering if this behavior is normal?
`
from learn2learn.utils import clone_module
class TestModule(nn.Module):
def __init__(self):
super(type(self), self).__init__()
cnn = [nn.Conv2d(3,32,3,2,1), nn.ReLU(), nn.Conv2d(32,32,3,2,1), nn.ReLU(), nn.Conv2d(32,32,3,2,1), nn.ReLU()]
self.seq = nn.Sequential(*cnn)
self.head = nn.Sequential(*[nn.Conv2d(32,32,3,2,1), nn.ReLU(), nn.Conv2d(32,100,3,2,1)])
self.net = nn.Sequential(self.seq, self.head)
def forward(self, x):
return self.net(x)
module = TestModule()
print(len(list(module.parameters())))
print([n for (n,p) in module.named_parameters()])
print(len(list(clone_module(module).parameters())))
print([n for (n,p) in clone_module(module).named_parameters()])
`
Result of this code shows that,
`
10
['seq.0.weight', 'seq.0.bias', 'seq.2.weight', 'seq.2.bias', 'seq.4.weight', 'seq.4.bias', 'head.0.weight', 'head.0.bias', 'head.2.weight','head.2.bias']
20
['seq.0.weight', 'seq.0.bias', 'seq.2.weight', 'seq.2.bias', 'seq.4.weight', 'seq.4.bias', 'head.0.weight', 'head.0.bias', 'head.2.weight','head.2.bias', 'net.0.0.weight', 'net.0.0.bias', 'net.0.2.weight', 'net.0.2.bias', 'net.0.4.weight', 'net.0.4.bias','net.1.0.weight', 'net.1.0.bias', 'net.1.2.weight', 'net.1.2.bias']
`
where the number of parameters increases, duplicating the parameters in module.net which is just a sequential module made for connecting the output of module.cnn and the input of module.head.
I suspect this mismatch is causing this warning message everytime I run my code?
'
/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py:385: UserWarning: The .grad attribute of a Tensor that is not a leaf Tensor is being accessed. Its .grad attribute won't be populated during autograd.backward(). If you indeed want the gradient for a non-leaf Tensor, use .retain_grad() on the non-leaf Tensor. If you access the non-leaf Tensor by mistake, make sure you access the leaf Tensor instead. See github.com/pytorch/pytorch/pull/30531 for more informations.
if param.grad is not None:
'
Thanks!
Hi, I've been using learn2learn 0.1.2 version. I was wondering if this behavior is normal?
`
`
Result of this code shows that,
`
`
where the number of parameters increases, duplicating the parameters in module.net which is just a sequential module made for connecting the output of module.cnn and the input of module.head.
I suspect this mismatch is causing this warning message everytime I run my code?
'
'
Thanks!