RuntimeError: cannot register a hook on a tensor that doesn't require gradient

[kevinxu-cn97]

my model is resnet18,and i change its fc,1000 to 2,
from torchvision import models model = models.resnet18(pretrained = False) model.fc = nn.Linear(512, 2) print(model)
after train,i try to use cam,

from torchcam.methods import SmoothGradCAMpp, GradCAMpp
cam_extractor = GradCAMpp(model)`

how ever:

RuntimeError                              Traceback (most recent call last)
/tmp/ipykernel_36/4220720489.py in <module>
      4 print()
      5 from torchcam.methods import SmoothGradCAMpp, GradCAMpp
----> 6 cam_extractor = GradCAMpp()

/opt/conda/lib/python3.7/site-packages/torchcam/methods/gradient.py in __init__(self, model, target_layer, input_shape, **kwargs)
     33     ) -> None:
     34 
---> 35         super().__init__(, target_layer, input_shape, **kwargs)
     36         # Init hook
     37         self.hook_g: List[Tensor] = [None] * len(self.target_names)  # type: ignore[list-item]

/opt/conda/lib/python3.7/site-packages/torchcam/methods/core.py in __init__(self, model, target_layer, input_shape, enable_hooks)
     52         elif target_layer is None:
     53             # If the layer is not specified, try automatic resolution
---> 54             target_name = locate_candidate_layer(, input_shape)  # type: ignore[assignment]
     55             # Warn the user of the choice
     56             if isinstance(target_name, str):

/opt/conda/lib/python3.7/site-packages/torchcam/methods/_utils.py in locate_candidate_layer(mod, input_shape)
     41     # forward empty
     42     with torch.no_grad():
---> 43         _ = mod(torch.rand(1, *input_shape, device=next(mod.parameters()).data.device))
     44 
     45     # Remove all temporary hooks

/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
   1069             input = bw_hook.setup_input_hook(input)
   1070 
-> 1071         result = forward_call(*input, **kwargs)
   1072         if _global_forward_hooks or self._forward_hooks:
   1073             for hook in itertools.chain(

/opt/conda/lib/python3.7/site-packages/torchvision/models/resnet.py in forward(self, x)
    247 
    248     def forward(self, x: Tensor) -> Tensor:
--> 249         return self._forward_impl(x)
    250 
    251 

/opt/conda/lib/python3.7/site-packages/torchvision/models/resnet.py in _forward_impl(self, x)
    238         x = self.layer2(x)
    239         x = self.layer3(x)
--> 240         x = self.layer4(x)
    241 
    242         x = self.avgpool(x)

/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
   1074                     _global_forward_hooks.values(),
   1075                     self._forward_hooks.values()):
-> 1076                 hook_result = hook(self, input, result)
   1077                 if hook_result is not None:
   1078                     result = hook_result

/opt/conda/lib/python3.7/site-packages/torchcam/methods/gradient.py in _hook_g(self, module, input, output, idx)
     51         """Gradient hook"""
     52         if self._hooks_enabled:
---> 53             self.hook_handles.append(output.register_hook(partial(self._store_grad, idx=idx)))
     54 
     55     def _backprop(self, scores: Tensor, class_idx: int) -> None:

/opt/conda/lib/python3.7/site-packages/torch/_tensor.py in register_hook(self, hook)
    287             return handle_torch_function(Tensor.register_hook, (self,), self, hook)
    288         if not self.requires_grad:
--> 289             raise RuntimeError("cannot register a hook on a tensor that "
    290                                "doesn't require gradient")
    291         if self._backward_hooks is None:

RuntimeError: cannot register a hook on a tensor that doesn't require gradient

[frgfm]

Hi @kevinxu-cn97 👋

That looks more like a bug report than a "Question & Answers" 😅
If you don't mind, could you open an issue instead and fill the bug report template? 🙏
(I spent time creating those templates so that we can solve quickly all types of problems, there's a lot of information missing from your message for me to debug this efficiently 😅 including your environment configuration, etc)

[kevinxu-cn97]

thank you and now the error is solved!(i made a mistic(^▽^))

[frgfm]

Oh glad to hear this!
Could you share what the mistake was please? 🙏 (so that next visitors encountering the same problem will see your solution)

[JACKCHAN000]

@kevinxu-cn97
I have the same problem too.
How do you fix it?🙏

[RuixiangZhao]

@JACKCHAN000
I met the same problem too. Have you solved this problem？
My model is resnet 50, replacing the fc layer.
Packages' version:

torch==1.10.0
torchvision==0.11.1
torchcam==0.3.2.dev0

[frgfm]

Hi all @kevinxu-cn97 @JACKCHAN000 @RuixiangZhao 👋

As discussed in #146, here are a few updates:

• so far, I haven't received enough information from you to debug this properly. I need to be able to either reproduce the bug or identify the source clearly to help you.
• Can any of you open a GH issue on this repo please? 🙏 There is a bug report template, just follow it, it will be much more helpful than a discussion, which is not exactly suited for this situation.
• If it was a mistake on your end, please share it with others, this is open-source, it's better when it goes both ways :)

One last thing to illustrate my struggle here:

1. I first tried to reproduce the initial problem reported by @kevinxu-cn97 here using this minimal runnable snippet (I insist, if your snippet uses a local file I cannot access, I can't run the snippet)

from torch import nn
from torchvision.models import resnet18
from torchcam.methods import GradCAMpp

# Create the model
model = resnet18(pretrained=False)
model.fc = nn.Linear(512, 2)

# Hook it
cam_extractor = GradCAMpp(model)

which runs perfectly on my end. According to your error, perhaps you've frozen layers of the model (and thus the snippet is incomplete), but I cannot infer much more 🤷‍♂️

2. Then I tried the following to reproduce the situation of @RuixiangZhao (cf. RuntimeError: cannot register a hook on a tensor that doesn't require gradient #146):

import torch
from torchvision import transforms
from torchvision.models import resnet50
from torchcam.methods import SmoothGradCAMpp
from PIL import Image

my_model = resnet50().eval()
cam_extractor = SmoothGradCAMpp(my_model)
img = Image.fromarray((255 * torch.rand((224, 224, 3))).round().to(dtype=torch.uint8).numpy())
test_transforms = transforms.Compose(
        [transforms.Resize(224),
        transforms.ToTensor(),
        transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])])
input_tensor = test_transforms(img)
out = my_model(input_tensor.unsqueeze(0))

which also runs perfectly on my end 🤷‍♂️

Let's get to the bottom of this together 🙌
Can someone please open a fully detailed Github issue about this? That should provide enough information to fix this quickly 👍

[RuixiangZhao]

@frgfm Thank you very much for your reply and your wonderful work!
I'm sorry for the incomplete description of my problem. I think I have found my mistake with your help.
Just like you said perhaps you've frozen layers of the model (and thus the snippet is incomplete), I was using pretrained resnet50 and just replacing the fc layer:

resnet50 = models.resnet50(pretrained=True)
for param in resnet50.parameters():
    param.requires_grad = False
fc_inputs = resnet50.fc.in_features
resnet50.fc = nn.Sequential(
    nn.Linear(fc_inputs, 256),
    nn.ReLU(),
    nn.Dropout(0.4),
    nn.Linear(256, num_classes),
    nn.LogSoftmax(dim=1)
)

[frgfm]

Glad to hear it helped @RuixiangZhao !
FYI, if you still prefer to freeze your model's layers, you can switch to an activation-based method from the library ;)
Alternatively, you can keep gradient-based methods, but then you can only freeze layers that come before your target_layer! (if you target layer4, you can freeze up until layer3)

[JACKCHAN000]

@frgfm
I follow your suggestion and switch it to the Activation-based methods.
https://frgfm.github.io/torch-cam/methods.html#activation-based-methods

It finally works now.
Thank you!!!

[richardbransonzp]

this website is '404 not found', could you tell us more details about this issue? thanks a lot!

[frgfm]

@richardbransonzp it looks like there was a formatting issue on the message of Jack! I just edited it, you should be able to navigate to the link now :)

[richardbransonzp]

thank you very much!

[IshaanShettigar]

Could you help me out with my implementation? I'm no expert at PyTorch but I believe I have frozen the required layers and kept the target layer unfrozen but I am still getting the same error as @kevinxu-cn97 . I have attached all relevant code snippets below.

This is the part where I import the model and freeze the layers, then change the model. fc after I have frozen the layers.

model_ft = models.resnet18(pretrained=use_pretrained)
        set_parameter_requires_grad(model_ft, feature_extract)
        num_ftrs = model_ft.fc.in_features
        model_ft.fc = nn.Linear(num_ftrs, num_classes)
        input_size = 224

let me show you the implementation of set_parameter_requires_grad(). I had set feature_extracting to True

def set_parameter_requires_grad(model, feature_extracting):
    if feature_extracting:
        for param in model.parameters():
            param.requires_grad = False

This is where I got my error

from torchcam.methods import LayerCAM, SmoothGradCAMpp
cam_extractor = SmoothGradCAMpp(model_ft)