Add MMLU downstream tasks

[OyvindTafjord]

Together with @yulinggu-cs, we added the following tasks from MMLU to the in-loop downstream evaluation suite:

Validation sets:

• mmlu_stem (321 qs)
• mmlu_humanities (518 qs)
• mmlu_social_sciences (337 qs)
• mmlu_other (355 qs)

Test sets:

• mmlu_stem_test (3018 qs)
• mmlu_humanities_test (4705 qs)
• mmlu_social_sciences_test (3077 qs)
• mmlu_other_test (3242 qs)

We also updated the metric_type for boolq and arc_challenge from "pmi_dc" to "acc" and "len_norm". These are generally correlated, although ideally "pmi_dc" would be used for several datasets once it's properly implemented. boolq should be fine with "acc" though.

Note that this adds some processing time at startup.

[OyvindTafjord]

There's one failing type check that I can't quite figure out, it doesn't like the dataset_name: Optional[Union[str, List[str]]] = None combined with the subsequent assignment.

[dirkgr]

That's all it takes? Is it because hails/mmlu_no_train already has the data in a workable format?

[OyvindTafjord]

The HF dataset (either hails/mmlu_no_train, or the equivalent, but larger with random training stuff, cais/mmlu) has instances in the format shown here, so just need to extract the appropriate pieces similar to other datasets (like ARC etc).

[dirkgr]

MMLU also needs to be added to the configs, right?