You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
hi, kindly helper,
I am a newer of lm-eval and I want to test the llama2 model with lm-evaluation-harness using wikitext-2 dataset. The setps are as follows:
Then I condigue the lm_eval parameters and execute the test using the command :
"lm_eval --model hf --model_args pretrained=~/LLM-Models/Llama-2-7b-hf --tasks wikitext --device cuda --batch_size 1 --output_path ./eval_harness/Llama-2-7b-hf-16b".
Then I get the results as follows:
hf-auto (pretrained=LLM-Models/Llama-2-7b-hf), gen_kwargs: (None), limit: None, num_fewshot: None, batch_size: 1
hi, kindly helper,
I am a newer of lm-eval and I want to test the llama2 model with lm-evaluation-harness using wikitext-2 dataset. The setps are as follows:
"lm_eval --model hf --model_args pretrained=~/LLM-Models/Llama-2-7b-hf --tasks wikitext --device cuda --batch_size 1 --output_path ./eval_harness/Llama-2-7b-hf-16b".
Then I get the results as follows:
hf-auto (pretrained=LLM-Models/Llama-2-7b-hf), gen_kwargs: (None), limit: None, num_fewshot: None, batch_size: 1
Are the results nomal? I think the value is too high compared with the test results of "https://github.com/ggerganov/llama.cpp?tab=readme-ov-file#quantization"
Conld anyone help me?
The text was updated successfully, but these errors were encountered: