Add CMB

[wangxidong06]

Motivation
Add CMB dataset.

Modification
Add /opencompass/opencompass/datasets/cmb.py
Modify /opencompass/opencompass/models/init.py
Add /opencompass/configs/datasets/cmb
Add /opencompass/data/CMB/test.json and /opencompass/data/CMB/val.json

[Leymore]

Thank you very much for your contribution. CMB is indeed a nice work. We're very sorry that we messed up the hash suffix of BBH last week, which resulted in us reopening an issue. Our apologies again.

I've noticed that your modifications involve the /opencompass/data/CMB/test.json and /opencompass/data/CMB/val.json files. However, these two files are added to the .gitignore and won't be pushed. Also, I noticed that this file doesn't exist on https://github.com/FreedomIntelligence/CMB. Therefore, you might want to consider sharing this file with us through other means, such as directly uploading it to the pull request, and then we can add this dataset to our releases.

Additionally, the meaning of PPL is to calculate the perplexity of different data prompt sentences. Given that there are multiple correct options in the CMB dataset, I'm not sure if a PPL evaluation would be feasible? If it's not feasible, you can directly delete the implementation of PPL. Otherwise, you might consider using PPL for single correct options and Gen for multiple correct options. You can refer to the following implementation:

https://github.com/open-compass/opencompass/blob/63ced828d86dcda4d11258f02492edc18c80aaa9/configs/datasets/agieval/agieval_mixed_2f14ad.py

[wangxidong06]

Thanks for your kindly advice. I remove the implementation of PPL and share the specific data to @gaotongxiao.

[gaotongxiao]

Thanks for your contribution!

[amulil]

@gaotongxiao @Leymore

FileNotFoundError: [Errno 2] No such file or directory: './data/CMB/test.json'

where I can download the test.json, I git pull the lastest code and download data from  https://github.com/FreedomIntelligence/CMB, there is no test.json file.