Llamatune fails with your example code from its home page

[IridiumMaster]

steps to reproduce

1. start a runpod container with the pytorch 2.01 template and lots of disk space
2. run your sample command on a properly formatted dataset:
   python -m llamatune.train
   --model_name meta-llama/Llama-2-13b-chat-hf
   --data_path master_qa.json
   --training_recipe lora
   --batch_size 8
   --gradient_accumulation_steps 4
   --learning_rate 1e-4
   --output_dir chat_llama2_13b
   --use_auth_token xxxzzz
3. result is:
   Model ready for training!
   trainable params: 250347520 || all params: 6922337280 || trainable: 3.616517223500557
   WARNING:root:Loading data...
   WARNING:root:Tokenizing inputs... This may take some time...
   config TrainingConfig(model_name='meta-llama/Llama-2-13b-chat-hf', data_path='master_qa.json', output_dir='chat_llama2_13b', training_recipe='lora', optim='paged_adamw_8bit', batch_size=8, gradient_accumulation_steps=4, n_epochs=3, weight_decay=0.0, learning_rate=0.0001, max_grad_norm=0.3, gradient_checkpointing=True, do_train=True, lr_scheduler_type='cosine', warmup_ratio=0.03, logging_steps=1, group_by_length=True, save_strategy='epoch', save_total_limit=3, fp16=True, tokenizer_type='llama', trust_remote_code=False, compute_dtype=torch.float16, max_tokens=4096, do_eval=True, evaluation_strategy='epoch', use_auth_token='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', use_fast=False, bits=4, double_quant=True, quant_type='nf4', lora_r=64, lora_alpha=16, lora_dropout=0.0)
   Traceback (most recent call last):
   File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
   return _run_code(code, main_globals, None,
   File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
   exec(code, run_globals)
   File "/usr/local/lib/python3.10/dist-packages/llamatune/train.py", line 50, in
   trainer.train()
   File "/usr/local/lib/python3.10/dist-packages/llamatune/trainer.py", line 25, in train
   self.model_engine.train(data_module=self.data_module)
   File "/usr/local/lib/python3.10/dist-packages/llamatune/model_engines/llama_model_engine.py", line 33, in train
   trainer = Trainer(
   File "/usr/local/lib/python3.10/dist-packages/transformers/trainer.py", line 405, in init
   raise ValueError(
   ValueError: The model you want to train is loaded in 8-bit precision. if you want to fine-tune an 8-bit model, please make sure that you have installed bitsandbytes>=0.41.1.

[jayantkhannadocplix1]

Hello @IridiumMaster
I encountered a similar problem and managed to resolve it by executing

pip install bitsandbytes==0.41.1

[IridiumMaster]

aye, that is the correct fix. Hoping the maintainers change their requirements.txt