Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

small problems when testing #7

Closed
csyanbin opened this issue Jul 27, 2017 · 2 comments
Closed

small problems when testing #7

csyanbin opened this issue Jul 27, 2017 · 2 comments

Comments

@csyanbin
Copy link

When I run the testing code after training as follow:
python main.py --datasource=miniimagenet --metatrain_iterations=60000 --meta_batch_size=4 --update_batch_size=1 --update_lr=0.01 --num_updates=5 --num_classes=5 --logdir=log/miniimagenet1shot/ --num_filters=32 --max_pool=True

python main.py --datasource=miniimagenet --metatrain_iterations=60000 --meta_batch_size=2 --update_batch_size=5 --update_lr=0.01 --num_updates=5 --num_classes=5 --logdir=logs/miniimagenet5shot/ --num_filters=32 --max_pool=True

I encountered several problems:
1). seems that in main.py Line 194-197 need indentation
L290 in main.py need to be commented for testing
2). I follow the default code and setting, but only get 46.90% and 61.03% for 1-shot and 5-shot rather than 48.70% and 63.11% reported in the paper on test set. Do I miss any training or testing tricks?

@cbfinn
Copy link
Owner

cbfinn commented Jul 27, 2017

For 1, actually lines 175-178 can just be deleted. Thanks for catching that.
For 2:
Are you using the validation set or the test set? [Test set performance tends to be higher than the val performance. To test on the test set, set --train=False --test_set=True]
Are you testing for more than 5 steps? I think I tested for ~8-10 gradient steps during meta-test time (exact number is in the paper).

@cbfinn
Copy link
Owner

cbfinn commented Jul 27, 2017

I also fixed the issue on line 290 (with an if statement)

@csyanbin csyanbin closed this as completed Aug 7, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants