Create a starter script for command-line training to integrate in pipelines etc...

[sebdg]

Add Starter Script for Fine-Tuning from command-line

Overview

This pull request introduces a starter script designed to help users fine-tune from the command-line or in DevOps/MLOps pipelines. The script includes configurable options for model loading, PEFT parameters, training arguments, and model saving/pushing functionalities. It serves as a comprehensive starting point, which users can customize according to their specific use case and requirements.

Key Features

• Model Loading and Configuration: Easily load and configure the unsloth models with options for sequence length, data type, and 4-bit quantization.
• PEFT Parameters: Configure PEFT model parameters including rank, LoRA alpha, dropout rate, bias settings, and gradient checkpointing.
• Training Arguments: Customize training settings such as batch size, gradient accumulation steps, warmup steps, maximum steps, learning rate, logging steps, optimizer type, weight decay, and learning rate scheduler type.
• Model Saving and Pushing: Options to save the trained model locally with specified quantization methods or push it to the Hugging Face Hub.

Customization

This script is intended to be a starting point. Users are encouraged to customize the following:

• Hyperparameters: Adjust learning rate, batch size, max steps, etc., to better suit your task and dataset.
• Dataset Loading and Preprocessing: Modify the dataset loading and preprocessing steps to align with your data.
• Model Saving and Pushing Configurations: Set the appropriate paths and tokens for saving or pushing the model.

Usage

Run the script with default settings or customize the arguments as needed:

python unsloth-cli.py --model_name "unsloth/llama-3-8b" --max_seq_length 8192 --dtype None --load_in_4bit \
--r 64 --lora_alpha 32 --lora_dropout 0.1 --bias "none" --use_gradient_checkpointing "unsloth" \
--random_state 3407 --use_rslora --per_device_train_batch_size 4 --gradient_accumulation_steps 8 \
--warmup_steps 5 --max_steps 400 --learning_rate 2e-6 --logging_steps 1 --optim "adamw_8bit" \
--weight_decay 0.005 --lr_scheduler_type "linear" --seed 3407 --output_dir "outputs" \
--report_to "tensorboard" --save_model --save_path "model" --quantization_method "f16" \
--push_model --hub_path "hf/model" --hub_token "your_hf_token"

To view all configurable options, use the help command:

python unsloth-cli.py --help

Contribution

Thank you for considering this contribution to the repository. I hope this script serves as a useful tool for the community in integrating their fine-tuning into larger automations and MLOps pipelines.

[bet0x]

@sebdg Would you mind adding environment variables support ?

[sebdg]

@bet0x sure i will see if I can do something for that will see to use dotenv too

[bet0x]

@sebdg Maybe reading a YAML so you load the variables from there ? 👯

[sebdg]

@sebdg Maybe reading a YAML so you load the variables from there ? 👯

Hi @bet0x, I intended to make the 'minimal' example as a kickstart, I'm afraid that adding all those will make this script much more complex and hence difficult to integrate/modify. maybe it's more something to be added to an examples repo/folder rather than in the main trunk as really the intend is more to give a kick starter than a polished all fits one cli.