Some questions about LoRA for pre-trained model

[iopwsy]

It is very excellect code and I have used it for almost anywork and it seems reduce my cost.
However, I still have some questions when using it, if possibile, I hope your response and it may benefit me a lot.

• I used it with my Net, it works well. However, when I used it on the pre-trained model, it raises the error "has no grad_fn", but if l used loss.requires_grad = True. Itr seems works well. I don't understand whether it is suitable? or make loss have grad means on LoRA linear or basic model?
• Another question: I use 90k data to finetune the pre-trained it seems no as good as fine tune directly. If I have enough data, how could I fine tune the model based on LoRA? I seem fail to find the suitable hyper-parameter.

I am new in Deeping Learning and these question confuse me recently, if possibile, I hope your answer and that may benefit me a lot.
Thanks a lot!

Best Wish!
Siyuan