You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Fine-tuning Salesforce/codegen-2B-mono on my own text to code Python dataset.
I quantized the model using HuggingFace transformer's BitsAndBytesConfig and LoRA config.
Implemented loRA config by using the following code:
from peft import LoraConfig, get_peft_model
lora_config = LoraConfig(
r=8,
lora_alpha=32,
target_modules=[],
lora_dropout=0.05,
bias="none",
task_type="CAUSAL_LM"
)
model = get_peft_model(model, lora_config)
I am not sure what should i be using in target models, in official git provided below.
Fine-tuning Salesforce/codegen-2B-mono on my own text to code Python dataset.
I quantized the model using HuggingFace transformer's BitsAndBytesConfig and LoRA config.
Implemented loRA config by using the following code:
I am not sure what should i be using in target models, in official git provided below.
Target modules for CodeGen is not mentioned.
peft git repo
Please help me out with this.
Any of your inputs will be highly appreciated.
Thank You!
The text was updated successfully, but these errors were encountered: