You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The original GPTQ version model is 3GB.
The created LoRA is 100MB (optimizer.pt + adapter_model.bin).
However, after merging LoRA, the size is 12.6G (pytorch_model-00001-of-00002.bin + pytorch_model-00002-of-00002.bin).
--load_in_4bit=True made no difference
Steps to reproduce
create lora from gptq model and merge with gptq model.
Please check that this issue hasn't been reported before.
Expected Behavior
If I create a LORA for a GPTQ model and merge it with the original GPTQ model, I should be able to create a model with approximately the same size.
Current behaviour
I used examples/llama-2/gptq-lora.yml.
After creating lora with reference to , I merged it with the following command.
The original GPTQ version model is 3GB.
The created LoRA is 100MB (optimizer.pt + adapter_model.bin).
However, after merging LoRA, the size is 12.6G (pytorch_model-00001-of-00002.bin + pytorch_model-00002-of-00002.bin).
--load_in_4bit=True made no difference
Steps to reproduce
create lora from gptq model and merge with gptq model.
Config yaml
Possible solution
No response
Which Operating Systems are you using?
Python Version
3.10
axolotl branch-commit
axolotl 0.3.0
Acknowledgements
The text was updated successfully, but these errors were encountered: