Where can I download the MMBench-test-CN and MMBench-dev-CN benchmark #448
-
|
Where can I download the MMBench-test-CN and MMBench-dev-CN benchmark |
Beta Was this translation helpful? Give feedback.
Replies: 3 comments 7 replies
-
|
Hi, @laserwave , |
Beta Was this translation helpful? Give feedback.
-
|
I am confused how the scores of MMBench-test(CN), MMBench-dev(CN) and CCBench-test on the leaderboard page （https://opencompass.org.cn/leaderboard-multimodal） are produced. Did the MLLMs use the same data? Or the results are produced by the official of OpenCompass? |
Beta Was this translation helpful? Give feedback.
-
|
Hey, laserwave, |
Beta Was this translation helpful? Give feedback.
-
Hi, I was wondering if the CN dev/test datasets were ready to release? |
Beta Was this translation helpful? Give feedback.
-
|
You can download the dataset here: |
Beta Was this translation helpful? Give feedback.
-
|
Thank you so much for sharing. Could I simply translate it as "有下列几个选项：" with the chinese colon. |
Beta Was this translation helpful? Give feedback.
-
|
Besides, I found the count of dataset after loading is slightly different: I found following index were missing in MMBench_CN_dev: |
Beta Was this translation helpful? Give feedback.
-
|
And I also found some text error in CN dataset, like in CN_test dataset: |
Beta Was this translation helpful? Give feedback.
-
Beta Was this translation helpful? Give feedback.
-
|
Resolved |
Beta Was this translation helpful? Give feedback.
Hi, @laserwave ,
We are preparing to release the dataset (basically, it's a translated Chinese version of MMBench with human verification). The dataset will be available (high likely) in next week.