Convert unsharded model to huggingface format

[wjurayj]

This script will convert a checkpoint fine-tuned in Lit-LLaMA into a set of files that you can load & use within huggingface.

[wjurayj]

Issue

[LamOne1]

Does the current code take into consideration adapter_v2 parameters like "lm_head.adapter_bias"?

[wjurayj]

Does the current code take into consideration adapter_v2 parameters like "lm_head.adapter_bias"?

No, this only works to convert the base model into a huggingface format. For a LoRA, you can just use the Lit-LLaMA script to convert it into a base model. Whereas for a different adapter, my understanding is that it should be converted to a separate huggingface module, like here.

[LamOne1]

No, this only works to convert the base model into a huggingface format. For a LoRA, you can just use the Lit-LLaMA script to convert it into a base model. Whereas for a different adapter, my understanding is that it should be converted to a separate huggingface module, like here.

is there a way to combine adapter weights with the base model then convert the whole model to AutoModels in transformers?

[wjurayj]

No, this only works to convert the base model into a huggingface format. For a LoRA, you can just use the Lit-LLaMA script to convert it into a base model. Whereas for a different adapter, my understanding is that it should be converted to a separate huggingface module, like here.

is there a way to combine adapter weights with the base model then convert the whole model to AutoModels in transformers?

This is possible when using the LoRA adapter, with the convert_lora_weights script and the instructions here. However, will be less straightforward when using a prefix approach like the LLaMA-adapter paper does because this changes the architecture of the model when you run inference on it.

As it stands, my script is only equipped to convert a model fine-tuned with the full weights or using a LoRA, since these will ultimately result in a model with the same architecture as the original LLaMA model. I'm sure that it's possible to write a similar script generalizes to prefix-tuned adapter models, but I believe this would be beyond the scope of what this script aims to accomplish.

[LamOne1]

Thank you @wjurayj for you help, I appreciate your help.

[LamOne1]

Dear @wjurayj ,
I'd appreciate your help in this issue

[LamOne1]

After convering my finetuned model with LoRa to huggingface using this code the model acted as a pretrained model rather than a finetuned one.

[wjurayj]

@awaelchli @carmocca @lantiga How can I get this contribition added to the codebase?