Question about parameter fixing

[c-box]

I notice that you add the whole word embedding to the optimizer instead of just prompts?

# Add word embeddings to the optimizer
optimizer = AdamW([{'params': model.base_model.embeddings.word_embeddings.parameters()}], lr=args.learning_rate, correct_bias=False)

Would the embedding of other words also be updated?

[a3616001]

Hi @c-box, thanks for your interest!

The embeddings of other words won't be updated, as we manually set the gradients of other word embeddings to be 0 here.

[c-box]

Oh, I missed it. Thank you very much!