New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Runtime error "Cannot set a non-string value as the PAD token" #4
Comments
I replaced the PAD token from 0 to '0' and solved this issue. Due to the resourse restricted, I replaced the basemodel to Llama2-13B to train and evaluate the E4SRec, but the gained result is very poor. torchrun --nproc_per_node=8 --master_port=1907 finetune.py \
--base_model /data06/Llama-2-13b-hf \
--data_path Beauty \
--task_type sequential \
--output_dir ./LLM4Rec \
--batch_size 16 \
--micro_batch_size 1 \
--num_epochs 1 \
--learning_rate 0.0003 \
--cutoff_len 4096 \
--val_set_size 0 \
--lora_r 16 \
--lora_alpha 16 \
--lora_dropout 0.05 \
--lora_target_modules '[gate_proj, down_proj, up_proj]' \
--train_on_inputs False \
--add_eos_token False \
--group_by_length False \
--prompt_template_name alpaca \
--lr_scheduler 'cosine' \
--warmup_steps 100 \
--wandb_project E4SRec The infer script is torchrun --nproc_per_node=8 --master_port=1234 inference.py \
--base_model /data06/Llama-2-13b-hf \
--data_path Beauty \
--task_type sequential \
--checkpoint_dir ./LLM4Rec/ \
--cache_dir cache_dir/ \
--output_dir ./LLM4Rec/ \
--batch_size 16 \
--micro_batch_size 1 The reported results is that Evaluation for User:
Precision@1: 8.9433439162903e-05
Recall@1: 8.9433439162903e-05
MRR@1: 8.9433439162903e-05
MAP@1: 8.9433439162903e-05
NDCG@1: 8.9433439162903e-05
Precision@5: 8.9433439162903e-05
Recall@5: 0.00044716719581451504
MRR@5: 0.0001974988448180775
MAP@5: 0.0001974988448180775
NDCG@5: 0.0002586554819987668
Precision@10: 8.943343916290303e-05
Recall@10: 0.0008943343916290301
MRR@10: 0.00025114116358582666
MAP@10: 0.00025114116358582666
NDCG@10: 0.0003971278706389283
Precision@20: 7.825425926754018e-05
Recall@20: 0.0015650851853508028
MRR@20: 0.0002965062025846168
MAP@20: 0.0002965062025846168
NDCG@20: 0.0005650891080215324
Precision@100: 7.154675133032246e-05
Recall@100: 0.007154675133032241
MRR@100: 0.00040016219874475093
MAP@100: 0.00040016219874475093
NDCG@100: 0.0015212688144377391 |
The similar issues here.I would like to ask if there are any issues with the evaluation? There are too many decimals,like 0.000. |
If you want to use a smaller base model, please use garage-bAInd/Platypus2-13B or garage-bAInd/Platypus2-7B instead. |
Dear authors,
Thanks for your nice work! I have already cloned this repo and downloaded the 'Platypus2-70B-instruct' model. However, I encounter (meet) with the following error when running the 'fine-turning.sh':
It seems like the token cannot be the number.
The text was updated successfully, but these errors were encountered: