Question about Table 12 in the paper

[yutaro-s]

Hi. Thanks for the great work!
I'm trying to reproduce the results of Table 12 (impact of expander dimensionality) in your paper.
Could you teach me what hyperparameters you used in the experiments?

[Adrien987k]

Hi,

All the hyper-parameters are described in Section 4.2 of the paper, except for the base learning rate for 100 epochs of pretraining which is 0.3 instead of 0.2 (Described in Appendix C.4 VICReg Setup.).
The code runs on 8 GPUs with an effective batch size of 2048.
The hyper-parameter are the same for every embedding size of Table 12.

Hope this helps.

[yutaro-s]

Thanks for your quick reply. I'll try it.