Cannot pass model.parameters() for variables in TorchModel fit_generator()

[gauthamk02]

🐛 Bug

When passing the variables for the optimizer to fit_generator() function in the TorchModel class, an error occurs if the variables object passed is the return object from model.parameters().

The error is ValueError: optimizer got an empty parameter list.

This happens because model.parameters() returns a generator object that's consumed and stored in a variable and the empty generator object later is passed to the optimizer. The fit_generator() snippet that causes the error is pasted below.

if variables is None:
    optimizer = self._pytorch_optimizer
    lr_schedule = self._lr_schedule
else:
    var_key = tuple(variables) # <-------------------- the generator gets used up here
    if var_key in self._optimizer_for_vars:
        optimizer, lr_schedule = self._optimizer_for_vars[var_key]
    else:
        optimizer = self.optimizer._create_pytorch_optimizer(variables) # <------ empty generator object is passed here
        if isinstance(self.optimizer.learning_rate,
                      LearningRateSchedule):
            lr_schedule = self.optimizer.learning_rate._create_pytorch_schedule(
                optimizer)
        else:
            lr_schedule = None
        self._optimizer_for_vars[var_key] = (optimizer, lr_schedule)

To Reproduce

Run the below code to reproduce the error:

from deepchem.models import TorchModel
import deepchem as dc
import torch.nn as nn
import numpy as np

class Model(nn.Module):
    def __init__(self):
        super(Model, self).__init__()
        self.dense1 = nn.Linear(100, 50)
        self.dense2 = nn.Linear(50, 1)

    def forward(self, x):
        x = nn.functional.relu(self.dense1(x))
        x = self.dense2(x)
        return x

class DC_TorchModel(TorchModel):
    def __init__(self):
        model = Model()
        loss = dc.models.losses.L2Loss()
        output_types = ['prediction']
        super(DC_TorchModel, self).__init__(model, loss, output_types)

    def fit(self, dataset):
        variables = self.model.parameters() # <-------------------------------- This won't work
        # variables = list(self.model.parameters())  # <---------------------- This works
        return self.fit_generator(self.default_generator(dataset), variables=variables)
    
X = np.random.rand(10, 100)
y = np.random.rand(10, 1)
dataset = dc.data.NumpyDataset(X, y)

model = DC_TorchModel()
model.fit(dataset)

Stack Trace:

  File "/home/gautham/Desktop/repos/deepchem-test/file1.py", line 35, in <module>
    model.fit(dataset)
  File "/home/gautham/Desktop/repos/deepchem-test/file1.py", line 27, in fit
    return self.fit_generator(self.default_generator(dataset), variables=variables)
  File "/home/gautham/Desktop/repos/deepchem/deepchem/models/torch_models/torch_model.py", line 405, in fit_generator
    optimizer = self.optimizer._create_pytorch_optimizer(variables)
  File "/home/gautham/Desktop/repos/deepchem/deepchem/models/optimizers.py", line 233, in _create_pytorch_optimizer
    return torch.optim.Adam(params,
  File "/home/gautham/anaconda3/envs/deepchem/lib/python3.9/site-packages/torch/optim/adam.py", line 45, in __init__
    super().__init__(params, defaults)
  File "/home/gautham/anaconda3/envs/deepchem/lib/python3.9/site-packages/torch/optim/optimizer.py", line 261, in __init__
    raise ValueError("optimizer got an empty parameter list")
ValueError: optimizer got an empty parameter list

Expected behavior

The fit_generator() function should work the same if passed a generator object or a parameter list/tuple.

Environment

• OS: PopOS 22
• Python version: 3.9
• DeepChem version: 2.7.2.dev
• PyTorch version: 2.1.2+cu121

[rbharath]

Thank you for finding this! This is a bit of an edge case we haven't considered. Could you perhaps join the discord https://discord.gg/MtSR8svQ to discuss or come by OH https://forum.deepchem.io/t/announcing-the-deepchem-office-hours/293? It may be good to talk this one through before deciding next steps

[gauthamk02]

Sure. Im already on the discord and will join the OH.