About BN parameters

[itongworld]

In your code I see that you fix the parameters of Batch Normalization after 1 epoch (10000 episodes), but when I remove the constraint (i.e., before training use model.train(), and before val/test use model.eval()), the performance will drop sharply.

Have you observed the same degradation w/o fixing the BN params? and why is that?

[WenbinLee]

Yes, we had observed the same degradation without fixing the BN parameters. This is the reason why we fix the parameters of Batch Normalization after 10000 episodes. We don't know the true reason of this phenomenon yet. We guess BN will affect the generalization performance of models. Actually, the research on BN is very hot recently. Thanks.

[itongworld]

Thanks a lot for your insightful analysis. And do you think other metric-based few-shot classification models, i.e., Protonet, can also benefit from this fixed-parameter setting?

[WenbinLee]

You are welcome. Yes, we had reimplemented Protonet based on the same framework, and we found this setting can gain about 1% performance improvement.