[BUG]: using peft to train prompts: RuntimeError: Expected 4-dimensional input for 4-dimensional weight XXX, but got 3-dimensional in

[HaixHan]

🐛 Describe the bug

when i use the train_peft_prompts.py to train the prompts, i occur this errors in this line(as shown in figure). However, i run the train_prompts.py successfully without using peft model. So who can tell me how to run train_peft_prompts.py successfully? thx a lot!!!

Environment

No response

[binmakeswell]

Hi @HaixHan peft support is a community example and didn't maintain by us. You can try our official example.

[HaixHan]

Hi @HaixHan peft support is a community example and didn't maintain by us. You can try our official example.

ok， i got it！ and i have another quesition.I found that the models saved after training were very large, whether in phase 1, phase 2 or phase 3. How can I save only the lora weights after each training? Which code should I change? look forward to your reply very much !