Skip to content

Commit

Permalink
update
Browse files Browse the repository at this point in the history
  • Loading branch information
Leymore committed Aug 25, 2023
1 parent 40a3fc0 commit f37becd
Show file tree
Hide file tree
Showing 2 changed files with 39 additions and 3 deletions.
4 changes: 2 additions & 2 deletions README_zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -329,13 +329,13 @@ unzip OpenCompassData.zip
python run.py --models hf_llama_7b --datasets mmlu_ppl ceval_ppl
```

OpenCompass 预定义了许多模型和数据集的配置,你可以通过 [工具](<>) 列出所有可用的模型和数据集配置。
OpenCompass 预定义了许多模型和数据集的配置,你可以通过 [工具](./docs/zh_cn/tools.md#ListConfigs) 列出所有可用的模型和数据集配置。

```bash
# 列出所有配置
python tools/list_configs.py
# 列出所有跟 llama 及 mmlu 相关的配置
python tools/list_config.py llama mmlu
python tools/list_configs.py llama mmlu
```

你也可以通过命令行去评测其它 HuggingFace 模型。同样以 LLaMA-7b 为例:
Expand Down
38 changes: 37 additions & 1 deletion docs/zh_cn/get_started.md
Original file line number Diff line number Diff line change
Expand Up @@ -98,9 +98,45 @@ python run.py --models opt_125m opt_350m --datasets siqa_gen winograd_ppl
# 列出所有配置
python tools/list_configs.py
# 列出所有跟 llama 及 mmlu 相关的配置
python tools/list_config.py llama mmlu
python tools/list_configs.py llama mmlu
```
其样例输出如下:
```text
+-----------------+-----------------------------------+
| Model | Config Path |
|-----------------+-----------------------------------|
| hf_llama2_13b | configs/models/hf_llama2_13b.py |
| hf_llama2_70b | configs/models/hf_llama2_70b.py |
| hf_llama2_7b | configs/models/hf_llama2_7b.py |
| hf_llama_13b | configs/models/hf_llama_13b.py |
| hf_llama_30b | configs/models/hf_llama_30b.py |
| hf_llama_65b | configs/models/hf_llama_65b.py |
| hf_llama_7b | configs/models/hf_llama_7b.py |
| llama2_13b_chat | configs/models/llama2_13b_chat.py |
| llama2_70b_chat | configs/models/llama2_70b_chat.py |
| llama2_7b_chat | configs/models/llama2_7b_chat.py |
+-----------------+-----------------------------------+
+-------------------+---------------------------------------------------+
| Dataset | Config Path |
|-------------------+---------------------------------------------------|
| cmmlu_gen | configs/datasets/cmmlu/cmmlu_gen.py |
| cmmlu_gen_ffe7c0 | configs/datasets/cmmlu/cmmlu_gen_ffe7c0.py |
| cmmlu_ppl | configs/datasets/cmmlu/cmmlu_ppl.py |
| cmmlu_ppl_fd1f2f | configs/datasets/cmmlu/cmmlu_ppl_fd1f2f.py |
| mmlu_gen | configs/datasets/mmlu/mmlu_gen.py |
| mmlu_gen_23a9a9 | configs/datasets/mmlu/mmlu_gen_23a9a9.py |
| mmlu_gen_5d1409 | configs/datasets/mmlu/mmlu_gen_5d1409.py |
| mmlu_gen_79e572 | configs/datasets/mmlu/mmlu_gen_79e572.py |
| mmlu_gen_a484b3 | configs/datasets/mmlu/mmlu_gen_a484b3.py |
| mmlu_ppl | configs/datasets/mmlu/mmlu_ppl.py |
| mmlu_ppl_ac766d | configs/datasets/mmlu/mmlu_ppl_ac766d.py |
+-------------------+---------------------------------------------------+
```
用户可以按照第一列中的名称去作为 `python run.py` 中 `--models` 和 `--datasets` 的传入参数。在数据集部分,相同名称但不同后缀的数据集一般表示意味着其提示词或评测方式是不一样的。
对于 HuggingFace 模型,用户可以直接通过命令行设定模型参数,而无需额外配置文件。例如,对于 `facebook/opt-125m` 模型,可以通过以下命令进行评测:
```bash
Expand Down

0 comments on commit f37becd

Please sign in to comment.