Skip to content
oobabooga edited this page Jun 13, 2024 · 26 revisions

What Works

Loader Loading 1 LoRA Loading 2 or more LoRAs Training LoRAs Multimodal extension Perplexity evaluation
Transformers ✅** ✅*
llama.cpp use llamacpp_HF
llamacpp_HF
ExLlamav2_HF
ExLlamav2 use ExLlamav2_HF
AutoGPTQ
AutoAWQ ? ? ?
HQQ ? ? ? ?

❌ = not implemented

✅ = implemented

* Training LoRAs with GPTQ models also works with the Transformers loader. Make sure to check "auto-devices" and "disable_exllama" before loading the model.

** Multi-LoRA in PEFT is tricky and the current implementation does not work reliably in all cases.

Clone this wiki locally