Validation set for CIFAR10

[Holim0711]

Thanks for your great work!

But it seems there is no validation set sampling part in this repository.

As far as I know, in general, 5,000 images are subtracted from the training set and used as validation.
[1] Official FixMatch, validation set size options, fixmatch
[2] A. Oliver, et al., "Realistic Evaluation of Deep Semi-Supervised Learning Algorithms"

I tried to find it in the datasets/ssl_dataset.py, but I couldn't.
May I know where the validation set part, or why it's not there? Thanks!

[qianlanwyd]

Google's repo and fixmatch paper do not actually use validation dataset, please read fixmatch paper and their codes more carefully.

[Holim0711]

Google's repo and fixmatch paper do not actually use validation dataset, please read fixmatch paper and their codes more carefully.

But in FixMatch paper, they say they followed standard SSL evaluation protocols of [A. Oliver et al., ...]
And the protocol specifies a validation set of 5000 samples. (see A. Oliver et al., Appendix B)

Also, google's fixmatch repo has the option --dataset=cifar10.{random_seed}@{#.labeled}-{#.validation}.
And you can find For validation, available sizes are 1, 5000 in the README.md of the repo.

Correct me if I'm wrong...
Thank you!

[qianlanwyd]

In Fixmatch repo Readme, they wrote :
We compute the median accuracy of the last 20 checkpoints in the paper, this is done through this code:

[Holim0711]

Ah, I get it now. Thank you for your answer so much!