Experience Streamlining

[rasbt]

Experience Streamlining

To streamline the experience for a general audience, without totally changing the experience, here are a few suggestions for changes (some of them are also discussed in separate issues/PRs). But this issue is to put it all in context.

So, below are

1. Improvement Suggestions: Things we would implement to streamline the experience to go from the "Current Experience" to the "Streamlined Experience"
2. "Current Experience": An example the 7 steps a user currently has to go through when finetuning with LoRA
3. "Streamlined Experience": the new 3-step experience after implementing all suggestions.

What do you think about this as a near-tearm improvement?

---

1) Improvement Suggestions

• 1. Convert downloaded HF checkpoint automatically upon download by default. Make it a flag to optionally disable though for debugging purposes, and keep the scripts/convert_hf_checkpoint.py for developers (but regular users don't have to worry about it.) (In progress via Automatically convert checkpoint files upon download #976)

• 2. Prepare the dataset automatically when running the finetuning script. (Addressed via Adrian's Dataset refactor in (4/n) Data Refactor - Finetuning Scripts #950 )

• 3. LoRA merges weights by default via a (--merge_weights true). But this can be disabled if users or developers prefer it. Defaulting to true removes one extra step when evaluating or using the resulting model. But let's keep the scripts/merge_lora.py if developers and users set --merge_weights false and prefer doing it manually.

• 4. Finetuning and pretraining scripts also copies the tokenizer files from the original checkpoint directory (needed for inference and evaluation) and the HF config.json (needed for converting back to HF format). (See Standardize checkpoints in lit-gpt #923 and Download config.json for conversion script #965). In progress via Copy configs to new checkpoint dir #984

---

---

2) Current Experience

0. Clone repository and set up requirements:

git clone https://github.com/Lightning-AI/lit-gpt.git
cd lit-gpt
pip install -r requirements-all.txt

1. Download a model of interest:

For convenience, we first specify an environment variable (optional) to avoid copy and pasting the whole path:

export repo_id=TinyLlama/TinyLlama-1.1B-intermediate-step-1431k-3T

Instead of using TinyLlama, you can replace the repo_id target with any other model repository
specifier that is currently supported by Lit-GPT. You can get a list of supported repository specifier
by running scripts/download.py without any additional arguments.

Then, we download the model we specified via $repo_id above:

python scripts/download.py --repo_id $repo_id

2. Convert the model into the Lit-GPT format:

python scripts/convert_hf_checkpoint.py --checkpoint_dir checkpoints/$repo_id

3. Prepare a dataset for finetuning:

python scripts/prepare_alpaca.py \
    --checkpoint_dir checkpoints/$repo_id \
    --destination_path data/alpaca

4. Finetune the model:

Set --train.epoch_size 100 for a quick dev run.

export finetuned_dir=out/lit-finetuned-model

python finetune/lora.py \
   --io.checkpoint_dir checkpoints/$repo_id \
   --io.train_data_dir data/alpaca \
   --io.val_data_dir data/alpaca \
   --train.epochs 1 \
   --train.epoch_size 100 \
   --io.out_dir $finetuned_dir

5. Merge LoRA weights:

Note that this step only applies if the model was finetuned with lora.py above and not when full.py was used for finetuning.

python scripts/merge_lora.py \
    --checkpoint_dir checkpoints/$repo_id \
    --lora_path $finetuned_dir/lit_model_lora_finetuned.pth \
    --out_dir $finetuned_dir/merged/

6. Copy files

cp checkpoints/$repo_id/tokenizer* $finetuned_dir/merged/
cp checkpoints/$repo_id/lit_config.json $finetuned_dir/merged/

7. Use model:

python chat/base.py --checkpoint_dir $finetuned_dir/merged/

---

---

3) Streamlined experience

0. Clone repository and set up requirements:

git clone https://github.com/Lightning-AI/lit-gpt.git
cd lit-gpt
pip install -r requirements-all.txt

1. Download a model of interest:

export repo_id=TinyLlama/TinyLlama-1.1B-intermediate-step-1431k-3T

python scripts/download.py --repo_id $repo_id \
  --convert_hf_checkpoint true

(By default, --convert_hf_checkpoint true.)

2. Finetune the model:

By default, weights are merged after finetuning via --merge_weights true. This creates a lit_model.pth in the output directory.

export finetuned_dir=out/lit-finetuned-model

python finetune/lora.py \
   --io.checkpoint_dir checkpoints/$repo_id \
   --data data.Alpaca \
   --train.epochs 1 \
   --train.epoch_size 100 \
   --io.out_dir $finetuned_dir \
   --merge_weights true

3. Use model:

python chat/base.py --checkpoint_dir $finetuned_dir/

[rasbt]

I think this is currently all addressed (there are only a few small enhancement regarding LoRA naming conventions).