Discussion : Steps for swapping Llama 2 with Llama 3

[rahulkrprajapati]

Hi @RenShuhuai-Andy ,
Great work on the paper. I'd been wondering about the accuracy improvement for real world use cases.
Would swapping Llama 2 with Llama 3 help with increasing the accuracy? And how can I go about executing and testing this out?

[RenShuhuai-Andy]

Hi, thanks for your interests.

I think replacing Llama 2 with Llama 3 needs re-conducting instruct tuning. You can try to replace the model path in https://github.com/RenShuhuai-Andy/TimeChat/blob/master/train_configs/stage2_finetune_time104k_valley72k.yaml#L12, https://github.com/RenShuhuai-Andy/TimeChat/blob/master/train_configs/stage2_finetune_time104k_valley72k.yaml#L54, and https://github.com/RenShuhuai-Andy/TimeChat/blob/master/train_configs/stage2_finetune_time104k_valley72k.yaml#L75 to the model path of Llama 3.