How was the fine-tunimg done exactly

[travellingsasa]

Hey there,

I am wondering how you did the fine-tuning here. You do not describe it in the paper.

Did you

1. Continue training starting from a pre trained model like open clip
2. Added a classification head and froze all other layer
3. Added a classification head and updated image and text encoders

I don't think you did 2 or 3 since you used full sentences as captions.

How did you do it?

All the best

[vinid]

Hi!!

It's contrastive fine-tuning, we use the same task CLIP was trained on. All unfrozen.

Let me know if you need more details!

[travellingsasa]

So when you say "same task CLIP was trained on" do I correctly assume you continued training without adding a classifier?

[vinid]

Yup, we keep the same contrastive pre-training objective

[travellingsasa]

Thank you for the clarification and the super quick reply :)

[vinid]

Happy to help!!

…

On Tue, Apr 30, 2024, 14:14 travellingsash ***@***.***> wrote: Thank you for the clarification and the super quick reply :) — Reply to this email directly, view it on GitHub <#32 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AARBSS6EQCFIX7EIF344DW3ZAACRRAVCNFSM6AAAAABG7C3GB2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAOBXGM2DEOJXGI> . You are receiving this because you commented.Message ID: ***@***.***>