Expand the size of CLIP

[Shuaizhang7]

Hello, thank you very much for your work. I would like to ask whether expanding the scale of pre-trained CLIP will help improve the final results? For example, change ViT-B-16 to ViT-L-14. After I changed CLIP to ViT-L-14, the results did not improve, so I don't know if there is a problem with my change or if this change is methodologically useless.

[timojl]

Thanks for your interest in our work. I'd expect ViT-L-14 to yield better results but hyperparameters might need to be changed. In particular, avoiding overfitting could be more challenging due to the internal dimension of 1024 in ViT-L.