[BUG] Discrepancy between patch and reload functionals in FLOPs profiler

[ameroyer]

In the Flops profiler, the list of functionals restored in _reload_functionals is missing some functions wrapped in _patch_functionals. This leads to wrong FLOPs counts when using consecutive calls to profiler.start_profile (and possibly some minor overhead when using these functional later on).

To reproduce
Example comparing torch.nn.LayerNorm (not properly restored) and torch.nn.Linear (properly restored): Repeated calls with the exact same inputs should give the same result.

modules_test = [torch.nn.LayerNorm(256), torch.nn.Linear(256, 128)]
num_calls = 3
for m in modules_test:
    print(f"\n\nTesting {m}")
    for i in range(num_calls):
        flops, macs, params = get_model_profile(m, (1, 256), print_profile=False)
        print(f"  Call {i + 1}: flops = {flops}; macs = {macs}; params = {params}")

giving the output:

Testing LayerNorm((256,), eps=1e-05, elementwise_affine=True)
  Call 1: flops = 1.28 K; macs = 0 MACs; params = 512
  Call 2: flops = 2.56 K; macs = 0 MACs; params = 512
  Call 3: flops = 3.84 K; macs = 0 MACs; params = 512


Testing Linear(in_features=256, out_features=128, bias=True)
  Call 1: flops = 65.54 K; macs = 32.77 KMACs; params = 32.9 k
  Call 2: flops = 65.54 K; macs = 32.77 KMACs; params = 32.9 k
  Call 3: flops = 65.54 K; macs = 32.77 KMACs; params = 32.9 k

Expected behavior
_reload_functionals should restore all functionals wrapped in _patch_functionals. It seems that (layer, group, instance)norm are missing, but I haven't checked the list carefully.

[ameroyer]

issue addressed/fixed in this thread: #2068