Clip text encoder instead of language model (GPT-2)

[jihwanp]

Hi thanks for providing great work!
Just in case, did you experiment with the text encoder of the clip instead of the language pretrained model?
Of course, I think the clip text encoder only learns the alignment with visual features, but I'm still curious whether clip text encoder has generation ability.

Thanks

[rmokady]

The textual encoder cannot generate text on its own
You can combine it with optimization or other component, but this is out of the scope of our work