-
Notifications
You must be signed in to change notification settings - Fork 94
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
How to apply LoRA adapter to a model loaded with OVModelForCausalLM()? #642
Comments
You probably can't do that once the model is loaded/exported to OpenVINO (or any framework with static computation graph), what you probably can do is to have one or many adapters fused into a base model and export it to OpenVINO.
|
Thank you for your detailed answer. I understood that dynamic switching is difficult with static graph frameworks such as OpenVINO. |
Additional question. |
for now it seems impossible to me, but my understanding of the OpenVINO runtime is still very narrow. |
In the transformers library, we can load multiple adapters to the original model by load_adapter then switch the specified adapter with set_adapter like below.
Now I want to apply LoRA adapters with OpenVINO, but I can't find an example of it.
Is it possible to do it with OVModelForCausalLM?
The text was updated successfully, but these errors were encountered: