[Feature Request] Return Number of Model Parameters

[isaykatsman]

Would be nice to call model.num_params() and be able to obtain the number of trainable parameters of a model. Especially useful during model summary printing. The PR should probably reference this: https://discuss.pytorch.org/t/finding-the-total-number-of-trainable-parameters-in-a-graph/1751/2

[soumith]

this can be trivially done with a line of code.

sum([p.numel() for p in model.parameters()])

I dont see any benefit of having an extra function for this.
I think this is more unnecessary because listing the number of parameters of a model is not something you repeatedly do 100s of times over your code. It's usually just a single line for logging purposes.

[dpappas]

This will not work for any model.

for example:

class MLP(nn.Module):
    def __init__(self, input_dim=None, sizes=None, activation_functions=None):
        super(MLP, self).__init__()
        ################################
        sizes               = [input_dim]+ sizes
        self.activations    = activation_functions
        self.linears        = []
        for i in range(len(sizes)-1):
            self.linears.append(nn.Linear(sizes[i], sizes[i+1]))
        ################################
    def forward(self, features):
        ret = features
        for layer, activation in zip(self.linears, self.activations):
            ret = layer(ret)
            if(activation is not None):
                ret = activation(ret)
        return ret

my_mlp = MLP(100, [10, 1], [None, None])
print(sum([p.numel() for p in model.parameters()]))

[soumith]

@dpappas that's because model.parameters() in your case will return an incomplete set of parameters. Do self.linears = nn.ModuleList() instead of []

[dpappas]

Perfect! Thank you very much.