[QUESTION] Difference 10-epochs, 40-epochs

[RGring]

Hi Ayush and Sayak,
I read your blog on wandb and find it interesting to see the performance on a relatively small dataset. Great work!

I am trying to reproduce your results with the official swAV-implementation. Though I only reach the 10-epochs plateau (~2.7).

I was wondering what are the difference between both runs? I found the following, did I miss any?

• projection to prototype architecture
• PolynomialDecay
• ... further?

Just curious, if you achieved the same results with the original pytorch implementation?

Thanks in advance :)

[sayakpaul]

Thank you for your interest @RGring.

Here are a couple of suggestions that the authors have put in the official repository and we have verified that ourselves and they work! We are yet to publish the changes but just wanted to confirm that those suggestions actually work.

Just curious, if you achieved the same results with the original pytorch implementation?

Actually, we did not run the official implementation so won't be able to lament.

I hope this helps.

[RGring]

Thanks for your answer. I've seen the suggestions and some of them helped, indeed. I guess, now it is a matter of hyperparameter-tuning. Looking forward to your new best score on the flower dataset :).

Thanks again!

[sayakpaul]

Yes, that is generally the case. Sometimes simplifying the data augmentation pipeline also helps quite a lot.

You can also take a look at SimSiam (https://github.com/sayakpaul/SimSiam-TF) which I open-sourced yesterday. It's by FAIR. It's way simpler and it provides pretty comparable results to SwAV, BYOL, MoCov2, and SimCLR.