getting statistics for batchnorm

[lmzintgraf]

Hi Chelsea,

I'm reimplementing MAML for few-shot classification, and I have troubles understanding how exactly you use batch norm. I was hoping you can help me out and clarify so I can understand this better. Below I'm assuming N=5-way k=1-shot learning and 4 tasks per meta-update.

During training, in the inner loop update, how do you compute the mean/var for batch normalisation? Do you

• use the (5x1)=5 images from the current batch, so the task-train set?
• use the (5x(1+15))=80 images from the current task, so the task-train and task-test set?
• use the (4x5x(1+15))=320 images from all tasks in the current meta-batch?

Also, does this differ depending on how many gradient update steps you do? And when you're evaluating, will you use the same procedure?

Thanks a lot in advance!

[cbfinn]

The batch statistics are always computed using the current batch. This does not differ with variable number of gradient steps, and it is the same during meta-training and meta-testing.

[lmzintgraf]

Thank you so much for the quick response!