calculate param problem

[WZMIAOMIAO]

Hi, I find that the results of fvcore.nn.prameter_count function and tensorflow are different when calculate same model.
The difference comes from the BN layer. tensorflow calculate beta, gamma, moving_mean and variance(four params), but fvcore.nn.prameter_count only calculate beta, gamma.

for example:

import torch
from fvcore.nn import parameter_count_table


def main():
    model = torch.nn.BatchNorm2d(1)
    print(parameter_count_table(model))


if __name__ == '__main__':
    main()

result:

| name    | #elements or shape   |
|:--------|:---------------------|
| model   | 2                    |
|  weight |  (1,)                |
|  bias   |  (1,)                |

only calculate beta, gamma.

In https://github.com/facebookresearch/fvcore/blob/master/fvcore/nn/parameter_count.py#L10-L30

r = defaultdict(int)
for name, prm in model.named_parameters():
    size = prm.numel()
    name = name.split(".")
    for k in range(0, len(name) + 1):
        prefix = ".".join(name[:k])

using model.named_parameters() only get beta, gamma no moving_mean and variance.

Are there any other considerations?

[ppwwyyxx]

In pytorch terminology, technically the moving mean and variance are "buffers" but not "parameters".

In general, it's hard to automatically decide for a module whether its "buffers" should be included in parameter count, since in many other modules they are not useful at all after training. Maybe an option can be added to control whether to include all buffers or no buffers in the count. Other suggestions are welcome.

[WZMIAOMIAO]

thank for your reply. Maybe adding an option is better.