Multi Adapter support #263

pacman100 · 2023-04-04T22:03:08Z

What does this PR do?

Adds multi-adapter training and inference support.
Fixes How to use multiple lora at the same time for text generation? #211, Is it possible to "unload" the PEFT LoRA weights after mutating the base model with PeftModel.from_pretrained ? #208, adds multiple adapters to a peft model #133 (comment), adds multiple adapters to a peft model #133 (comment) and https://gist.github.com/philschmid/821c5317d144250feef517aecd390b98

Usage

While loading the first adapter via PeftModel.from_pretrained, you can give it a name using **adapter_name** parameter. Else the default adapter name default is used.
To load another adapter, use **load_adapter()** method of PeftModel, e.g., model.load_adapter(peft_model_path, adapter_name)
To switch between adapters, use **set_adapter()** method of PeftModel, e.g., model.set_adapter(adapter_name)
To disable adapters, use context manager disable_adapter(), e.g., with model.disable_adapter()
Specific to LoRA method: To merge and unload the current active adapter so that the lora weights are added to the base model weights and he injected models are removed to get back the transformers base model with lora weights added, use merge_and_unload() method, e.g., model = model.merge_and_unload()

from peft import PeftModel
from transformers import LlamaTokenizer, LlamaForCausalLM, GenerationConfig

model_name = "decapoda-research/llama-7b-hf"
tokenizer = LlamaTokenizer.from_pretrained(model_name)
model = LlamaForCausalLM.from_pretrained(
    model_name,
    load_in_8bit=True,
    device_map="auto",
    use_auth_token=True
)
model = PeftModel.from_pretrained(model, "tloen/alpaca-lora-7b", adapter_name="eng_alpaca")
model.load_adapter("22h/cabrita-lora-v0-1", adapter_name="portuguese_alpaca")

model.set_adapter("eng_alpaca")
instruction = "Tell me about alpacas."
print(evaluate(instruction))

model.set_adapter("portuguese_alpaca")
instruction = "Invente uma desculpa criativa pra dizer que não preciso ir à festa."
print(evaluate(instruction))

with model.disable_adapter():
    instruction = "Invente uma desculpa criativa pra dizer que não preciso ir à festa."
    print(evaluate(instruction))

Link to colab notebook: https://colab.research.google.com/drive/1vrVg8G7AIdCM9qpcfZya0B7OEsPSAxbO?usp=sharing

Might have breaking changes

HuggingFaceDocBuilderDev · 2023-04-05T07:13:42Z

The documentation is not available anymore as the PR was closed or merged.

Dentoty · 2023-04-05T08:56:46Z

Would this work for Inference using LoRAs that were trained by using train_dreambooth.py ?

younesbelkada

Awesome work! 🔥 This feature is really great
I left a couple of open questions, just to confirm if I have understood some changes correctly, my main concern was whether this introduced some breaking changes with respect to adapters that are on the Hub, but I think that you are dealing correctly with that using ModulesToSaveWrapper
IMO we just need to figure out why some tests are failing, they're related to merging layers for some models, I made a suggestion below (I think in the test we don't deal correctly with the case merge_weights=False.
Also I would like to hear your thoughts on dropping MergedLinear in favor of a single Linear class!
Let's also introduce slow tests, (based on the snippet you shared), this can be done in #256

younesbelkada · 2023-04-05T10:16:16Z

src/peft/tuners/lora.py

+        if not self.merge_weights:
+            warnings.warn("Nothing to merge. Set merge_weights to True to enable merging.")
+            return


Maybe that could explain why the tests are failing, we should always call merge_weights=False (i.e. not test the case wheremerge_weights=False) --> or maybe call .eval() in case merge_weigths=True-

src/peft/tuners/lora.py

younesbelkada · 2023-04-05T10:17:51Z

src/peft/utils/other.py

+class ModulesToSaveWrapper(torch.nn.Module):
+    def __init__(self, module_to_save, adapter_name):
+        super().__init__()
+        self.original_module = module_to_save
+        self.modules_to_save = torch.nn.ModuleDict({})
+        self.update(adapter_name)
+        self.active_adapter = adapter_name
+
+    def update(self, adapter_name):
+        self.modules_to_save.update(torch.nn.ModuleDict({adapter_name: copy.deepcopy(self.original_module)}))
+
+    def forward(self, *args, **kwargs):
+        if self.active_adapter not in self.modules_to_save:
+            return self.original_module(*args, **kwargs)
+        return self.modules_to_save[self.active_adapter](*args, **kwargs)
+
+
+def _get_submodules(model, key):
+    parent = model.get_submodule(".".join(key.split(".")[:-1]))
+    target_name = key.split(".")[-1]
+    target = model.get_submodule(key)
+    return parent, target, target_name
+
+
+def _set_trainable(model, adapter_name):
+    key_list = [key for key, _ in model.named_modules()]
+    for key in key_list:
+        target_module_found = any(key.endswith(target_key) for target_key in model.modules_to_save)
+        if target_module_found:
+            parent, target, target_name = _get_submodules(model, key)
+            if isinstance(target, ModulesToSaveWrapper):
+                target.update(adapter_name)
+            else:
+                for param in target.parameters():
+                    param.requires_grad = True
+                setattr(parent, target_name, ModulesToSaveWrapper(target, adapter_name))
+
+
+def _set_adapter(model, adapter_name):
+    for module in model.modules():
+        if isinstance(module, ModulesToSaveWrapper):
+            module.active_adapter = adapter_name


I see, this avoids the breaking change I believe, can you confirm? 🙏

This avoids breaking changes when there are additional trainable layers such as classfier head/regression head on top of the model for tasks like AutoModelForSequenceClassification or TRL reward model head ... these are also saved along with the adapter weights and if each checkpoint has its own additional trainable layers, this makes sure that they are properly being called.

Awesome, this is great then!

younesbelkada · 2023-04-05T10:20:58Z

src/peft/peft_model.py

@@ -571,7 +645,8 @@ def forward(
        return_dict=None,
        **kwargs,
    ):
-        if not isinstance(self.peft_config, PromptLearningConfig):
+        peft_config = self.active_peft_config


IMO it should be documented somewhere that the way retrieving the peft config has changed, now it's active_peft_config rather than peft_config

younesbelkada

Looking great! Let's address the conversion script + example script / notebook in a follow up PR 🔥 Thanks for explaining in detail the approaches you made

lxe · 2023-04-07T00:18:58Z

Great timing on this! Tested it and it's working swimmingly ^^ 🤗

llllllim · 2024-01-02T11:18:06Z

How can I just upload the lora and remain the base model？I want to reduce memory usage

HyperdustLab · 2024-06-15T01:21:41Z

this is nice

pacman100 added 21 commits March 28, 2023 18:56

multi adapter for training and inference

c21afbe

Might have breaking changes

Update peft_model.py

af252b7

Update peft_model.py

7d7c598

Update lora.py

64cae2a

Update lora.py

e9d45da

Update lora.py

8ec7cb8

Update lora.py

090d074

Update peft_model.py

7c8ee58

fix bugs

002da1b

Merge branch 'main' into smangrul/multi-lora-support

6e0f124

fix 🐛

bd80d61

Update lora.py

96ca100

fix 🐛

d4b64c8

fixing 🐛

18ccde8

😅. Fix 🐛

122f708

fix more 🐛

d4c2bc6

😅

41b2fd7

fix more 🐛

dbdb8f3

😅

6f1f26f

😅

b9433a8

fix 🐛

7513195

pacman100 requested a review from younesbelkada April 4, 2023 22:03

pacman100 marked this pull request as ready for review April 4, 2023 22:03

Update config.py

44f3e86

pacman100 changed the title ~~Smangrul/multi lora support~~ Multi Adapter support Apr 4, 2023

fix doc failure

405f68f

Merge branch 'main' into smangrul/multi-lora-support

7888f69

younesbelkada reviewed Apr 5, 2023

View reviewed changes

fix test

37e1f9b

younesbelkada approved these changes Apr 5, 2023

View reviewed changes

pacman100 added 13 commits April 6, 2023 19:05

Merge branch 'main' into smangrul/multi-lora-support

75808eb

making adalora compatible with multiple adapters

7397160

Merge branch 'main' into smangrul/multi-lora-support

1a6151b

😅

74e2a3d

🐛 fixing

b728f5f

Update adalora.py

dee2a96

😅

b6c7514

fix 🐛

07a4b8a

fix

3aaf482

final fix I guess

a591b4b

fix 🐛

3258b70

fixing 🐛

d5feb8b

fixing adalora saving and loading

e8b0085

pacman100 merged commit 445940f into main Apr 6, 2023

lxe mentioned this pull request Apr 6, 2023

Full rework: Version 2 release lxe/simple-llm-finetuner#37

Merged

lxe mentioned this pull request Apr 7, 2023

Multi-adapter PEFT loading oobabooga/text-generation-webui#853

Closed

pacman100 mentioned this pull request Apr 7, 2023

model.load_adapter Problem #276

Closed

GanymedeNil mentioned this pull request Apr 7, 2023

How to load multiple LORAs while adjusting the weight of different LORAs #280

Closed

mcmonkey4eva mentioned this pull request Apr 14, 2023

initial multi-lora support oobabooga/text-generation-webui#1103

Merged

ziwang-com mentioned this pull request May 21, 2023

如何在调整不同 LORA 的重量的同时加载多个 LORA ziwang-com/zero-lora#34

Open

ChrisIsKing mentioned this pull request Jun 4, 2023

Support for swapping Dreambooth LoRAs when model is loaded in memory. #266

Closed

bohyunshin mentioned this pull request Jul 27, 2023

PEFT for Multiple Choice? #756

Closed

younesbelkada mentioned this pull request Aug 23, 2023

MergedLinear still available? #852

Closed

whybeyoung mentioned this pull request Sep 28, 2023

Support multi-adpater concurrent inferencing #973

Closed

Fraudsterrrr mentioned this pull request Nov 9, 2023

调用lora时能否同时调用多个lora chatchat-space/Langchain-Chatchat#2001

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Multi Adapter support #263

Multi Adapter support #263

pacman100 commented Apr 4, 2023

HuggingFaceDocBuilderDev commented Apr 5, 2023 •

edited

Loading

Dentoty commented Apr 5, 2023

younesbelkada left a comment •

edited

Loading

younesbelkada Apr 5, 2023

younesbelkada Apr 5, 2023

pacman100 Apr 5, 2023

younesbelkada Apr 5, 2023

younesbelkada Apr 5, 2023

younesbelkada left a comment

lxe commented Apr 7, 2023 •

edited

Loading

llllllim commented Jan 2, 2024

HyperdustLab commented Jun 15, 2024

Multi Adapter support #263

Multi Adapter support #263

Conversation

pacman100 commented Apr 4, 2023

What does this PR do?

Usage

HuggingFaceDocBuilderDev commented Apr 5, 2023 • edited Loading

Dentoty commented Apr 5, 2023

younesbelkada left a comment • edited Loading

Choose a reason for hiding this comment

younesbelkada Apr 5, 2023

Choose a reason for hiding this comment

younesbelkada Apr 5, 2023

Choose a reason for hiding this comment

pacman100 Apr 5, 2023

Choose a reason for hiding this comment

younesbelkada Apr 5, 2023

Choose a reason for hiding this comment

younesbelkada Apr 5, 2023

Choose a reason for hiding this comment

younesbelkada left a comment

Choose a reason for hiding this comment

lxe commented Apr 7, 2023 • edited Loading

llllllim commented Jan 2, 2024

HyperdustLab commented Jun 15, 2024

HuggingFaceDocBuilderDev commented Apr 5, 2023 •

edited

Loading

younesbelkada left a comment •

edited

Loading

lxe commented Apr 7, 2023 •

edited

Loading