使用案例指令评估LLaMA-7b模型在MMLU和C-Eval数据集上的性能时评测结果为空 #590
Unanswered
liujunfei678
asked this question in
Q&A
Replies: 2 comments 2 replies
-
Please check the log in *.out file |
Beta Was this translation helpful? Give feedback.
2 replies
-
您好,您遇到的这个问题解决了吗? |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
案例:
按照上述步骤确保OpenCompass正确安装并准备好数据集后,您可以使用以下命令评估LLaMA-7b模型在MMLU和C-Eval数据集上的性能:
python run.py --models hf_llama_7b --datasets mmlu_ppl ceval_ppl
结果:
![image](https://private-user-images.githubusercontent.com/134382914/282708559-7e39d88c-1d57-4937-84ed-ec7be4a31c5c.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MTg4MjcxMjIsIm5iZiI6MTcxODgyNjgyMiwicGF0aCI6Ii8xMzQzODI5MTQvMjgyNzA4NTU5LTdlMzlkODhjLTFkNTctNDkzNy04NGVkLWVjN2JlNGEzMWM1Yy5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjQwNjE5JTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI0MDYxOVQxOTUzNDJaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT0wODVmYTlkYzNmYjUyMzg5YTk1ZTNhYTAwYjk0MGYyOWQxZmFjMjhkN2M3ZTVkYjQ2N2NjNWJkMzI0ZjQ0YTg0JlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCZhY3Rvcl9pZD0wJmtleV9pZD0wJnJlcG9faWQ9MCJ9.w1U4I5SJjIt-Zn8LIEQANZmWzCiV-KBNyqfofe-felE)
过程中存在的报错:
11/14 15:44:18 - OpenCompass - ERROR - /home/liujunfei/opencompass/opencompass/runners/base.py - summarize - 63 - OpenICLInfer[llama-7b-hf/lukaemon_mmlu_professional_law_0] failed with code 1
11/14 15:44:18 - OpenCompass - ERROR - /home/liujunfei/opencompass/opencompass/runners/base.py - summarize - 63 - OpenICLInfer[llama-7b-hf/lukaemon_mmlu_professional_law_1] failed with code 1
我的问题:
我跟着opencompass的安装指南一步步操作下来,不知道为什么出现了** 评测结果为空**,且出现了一些我不理解的报错消息。
Beta Was this translation helpful? Give feedback.
All reactions