Collie

	Collie Herding Sheep, Collie Finetune LLaMa. 🐕 🌿 🐑, 🐕‍🦺 🛠️ 🦙

Getting started

install dependencies

pip install requirements.txt
# if dev
pip install dev-requirements.txt

prepare data

• jsonl format

{"instruction": "<instruction text>", "output": "<output text>"}
{"instruction": "<instruction text>", "output": "<output text>"}
{"instruction": "<instruction text>", "output": "<output text>"}

also support csv format

single gpu finetune

# get help
python finetune.py --help
# run finetune
# eg: python fineune.py BelleGroup/BELLE-7B-2M data.jsonl
python finetune.py <model_name_or_path> <data_file>

multi-gpu finetune

1. create config file

accelerate config --config_file <config_file>

2. run finetune script

accelerate launch --config_file <config_file> finetune.py <model_path> <data_file>

Dev

# run test case
make test
# check code style & lint code
make lint

TODO

• github action
• chat format data
• instrucion testing
• lora finetune