LISA Finetuning Example

[Jasonzzt]

Description

• add an example of new fine-tuning algorithm on IPEX-LLM llama2 7b
  #nano

• 'BF16Linear' object has no attribute 'enable_xetla', so 'enable_xetla' works only when q_proj.qtype == SYM_INT4/FP8E5

2. User API changes

How to run

python ./lisa_finetuning.py \
    --micro_batch_size 8 \
    --batch_size 128 \
    --base_model "/home/wangruonan/llm-models/Llama-2-70b-chat-hf" \
    --data_path "./alpaca_data_cleaned.json" \
    --output_dir "./ipex-llm-lisa-alpaca" \
    --gradient_checkpointing True \
    --lisa_activated_layers 1 \
    --lisa_interval_steps 20

3. Summary of the change

LISA Finetuning Example

4. How to test?

• N/A
• Unit test
• Application test
• Document test
• ...

[Uxito-Ada]

Are you sure other dtypes all do not need xetla?
maybe change else to elif BF16

[Jasonzzt]

Are you sure other dtypes all do not need xetla? maybe change else to elif BF16

Here is the supported qtype of xetla.

supported_qtype = self.q_proj.qtype == SYM_INT4 and full_attn
supported_qtype = supported_qtype or self.q_proj.qtype == FP8E5

On XPU, the method returns True when q_proj.qtype is among supported_qtypes(SYM_INT4/FP8E5) and enable_xetla is True.

ipex-llm/python/llm/src/ipex_llm/transformers/models/llama.py

Line 316 in 0d518aa

return device.type == "xpu" and enable_xetla and supported_qtype

So if qtype is other types, just set enable_xetla to False.

[Uxito-Ada]

lgtm