Training hyperparameters

[lhfowl]

Hello,

I'm hoping to confirm that the hyperparameters specified in your paper are correct. Specifically, for miniimagenet, 100k meta steps were taken during training? I ask because it seems some of the default values in the code are different.

[unixpickle]

See the readme: https://github.com/openai/supervised-reptile/blob/master/README.md#reproducing-training-runs which describes how to run the experiments by specifying the correct arguments

[siavash-khodadadeh]

In the readme file train-shots is 10, however, it seems that this should be 5-way 1-shot. Am I missing something?

transductive 1-shot 5-way Omniglot.

python -u run_omniglot.py --shots 1 --inner-batch 10 --inner-iters 5 --meta-step 1 --meta-batch 5 --meta-iters 100000 --eval-batch 5 --eval-iters 50 --learning-rate 0.001 --meta-step-final 0 --train-shots 10 --checkpoint ckpt_o15t --transductive

[siavash-khodadadeh]

Does this mean that during meta-learning we are training with 5-way 10-shot? but for the test, we evaluate on 5-way, 1-shot?

[MrDavidG]

@siavash-khodadadeh I have the same question. And I notice that the paper said

If we are doing K-shot, N-way classification, then we sample tasks by selecting N classes from C and then selecting K + 1 examples for each class. We split these examples into a training set and a test set, where the test set contains a single example for each class. 

It is a different setting with maml in evalution, maml use k_qry=15 (that is 15 examples for each class) to evaluate itself. It seems like the comparison in the experiment is unfair?