Finetuning with limited Labels ?

[rohit-gupta]

Hi as part of a study I am evaluating the capabilities of SSL models to learn from limited labels. Since ImageNet-1% and 10% checkpoints are not provided, I used the finetuning script to finetune on these image subsets. I was able to get 38% and 65% accuracy respectively with 1% and 10% labels using the Tiny variant. The results are probably suboptimal, so I am trying to figure out which hyperparamaters to tweak to improve them. Tweaking the learning rate doesn't seem to make much of a difference. Any suggestions on how to finetune on limited labels ?