How can I get the transformers' parameters?

[YongtaoGe]

❓ Questions & Help

Hi, I am new to transformers. Does this library offer an interface to compute the total number of different model's parameters?

[LysandreJik]

The models we use inherit directly from torch.nn.Module for our pytorch models and tf.keras.layers.Layer for tensorflow modules. You can therefore get the total number of parameters as you would do with any other pytorch/tensorflow modules:

sum(p.numel() for p in model.parameters() if p.requires_grad) for pytorch and
np.sum([np.prod(v.shape) for v in tf.trainable_variables]) for tensorflow, for example.

[YongtaoGe]

Got it!