Error when some layers are frozen

[arunpatala]

When I am trying to finetune a pretrained network, I am freezing some layers params using require_grad=False. The optimizer is trying to optimize all params causing
ValueError: optimizing a parameter that doesn't require gradients. Is there are a way to only pass params that have require_grad as True.
Thanks

[arunpatala]

I set the arg optimizer_parameters=model.fc.parameters() in trainer.compile.
But i guess it is being passed to optimizer initialization:

line 133, in set_optimizer
TypeError: init() got an unexpected keyword argument 'parameters'

[arunpatala]

added kwargs.pop('parameters',None) after line 128 and it works as expected

[ncullen93]

Oh I see. Wait can you tell me where in this function you added that line (i'd appreciate it):

    def set_optimizer(self, optimizer, **kwargs):
        if type(optimizer) is type or isinstance(optimizer, str):
            if 'parameters' in kwargs:
                parameters = kwargs['parameters']
            else:
                parameters = self.model.parameters()

            optimizer = _validate_optimizer_input(optimizer)
            self._optimizer = optimizer(parameters, **kwargs)
        else:
            self._optimizer = optimizer

[recastrodiaz]

I guess there is a better way, but the following works for me:

# 1. Set some layers of the model to be non-trainable (param.requires_grad = False)
# 2. Monkey patch the instance parameters() method to return trainable weights only
import types

def parameters(self):
    p = filter(lambda p: p.requires_grad, nn.Module.parameters(self))
    return p

model.parameters = types.MethodType(parameters, model)

# 3. Profit
trainer.fit_loader(train_loader, val_loader=val_loader, nb_epoch=1)

[arunpatala]

def set_optimizer(self, optimizer, **kwargs):
    if type(optimizer) is type or isinstance(optimizer, str):
        if 'parameters' in kwargs:
            parameters = kwargs['parameters']
            kwargs.pop('parameters',None)