torch::jit::script::Module has no zero_grad()

[peastman]

🐛 Bug

I'm writing a C++ program that loads a module from disk and tries to work with it. Unlike nn::Module, it appears that torch::jit::script::Module has no zero_grad() method. This makes it impossible to do optimization.

Environment

• PyTorch Version (e.g., 1.0): Nightly build downloaded Sept. 29, 2019
• OS (e.g., Linux): Linux
• How you installed PyTorch (conda, pip, source): Downloaded from https://download.pytorch.org/libtorch/nightly/cu101/libtorch-cxx11-abi-shared-with-deps-latest.zip
• Build command you used (if compiling from source): n/a
• Python version: n/a
• CUDA/cuDNN version: n/a
• GPU models and configuration: n/a
• Any other relevant information:

cc @suo

[suo]

Thanks for the report! We'll discuss this item and get back to you

[jamesr66a]

@peastman Can you manually loop over the parameters and zero them out as a workaround?

[peastman]

Thanks! It looks like the code to do that should be something like this:

for (auto param : module.get_parameters())
    if (param has a gradient)
        param.value().toTensor().grad().zero_();

I'm stuck on the second line. In Python I could just write grad is not None, but I can't figure out the corresponding C++.

[jhhurwitz]

@peastman, don't know if you are still having this problem, but one way to solve this would be to use an optimizer (e.g. torch::optim::SGD), which has a function zero_grad() for zeroing all the parameters in the optimizer.

Alternatively, here is a code snippet that loops over parameters and checks to see if they require a gradient as of the nightly build for 12/13/19.

    for (const auto& parameter : module->named_parameters(false)) {
        if (parameter.value.requires_grad()) {
         ...
        }
    }