Prompt tuning label embedding

[Even-ok]

Why is the label embedding in stage2 initialized by nn.embedding instead of CLIP text_encoder?

self.token_embedding = nn.Embedding(args.vocab_size, args.transformer_width)


self.label_emb = torch.zeros((len(self.name_lens), max(self.name_lens), self.transformer_width)).to(self.device)  
for i, embed in enumerate(self.token_embedding(self.label_token)):  
    self.label_emb[i][:self.name_lens[i]] = embed[4:4+self.name_lens[i]].clone().detach()  

[sunanhe]

Hi,

Thanks for your attention and problem. During the second stage, the token embeddings of label text are fixed. So, we use a nn.Embedding to save them.

I will close the issue. If you still have any problems, plz tell me.

Best,
Sunan.