Fine-tune LLaMA 2 (7B) with LoRA on meta-math/MetaMathQA Dataset

Fine-tuning and Inference codes on the MetaMath Dataset

Model Details

MetaMath-Fine-Tune-with-LoRA is trained to reason and answer mathematical problems on meta-math/MetaMathQA dataset. We used meta-llama/Llama-2-7b-hf as a base model and used LoRA to fine-tune it.

Model Description

• Project GitHub Page: https://github.com/SuperBruceJia/MetaMath-Fine-Tune-with-LoRA
• Developed by: Shuyue Jia in December, 2023
• Funded by: Boston University SCC
• Model type: fine-tuned
• Language(s) (NLP): English
• Finetuned from model: meta-llama/Llama-2-7b-hf

Results on GSM8K

Epoch	Accuracy on the testing set	Model Link
$1$	$0.609$	🤗 Hugging Face
$2$	$0.635$	🤗 Hugging Face
$3$	$0.641$	🤗 Hugging Face
$4$	$0.641$	🤗 Hugging Face

Deployment

# Load the Pre-trained LoRA Adapter
model.load_adapter("shuyuej/metamath_lora_llama2_7b_4_epoch")
model.enable_adapters()