Add MMLU evaluation script for Gemma models

[Konikz]

Add MMLU Evaluation Script

Description

This PR adds an MMLU evaluation script to address #36, providing a standardized way to evaluate Gemma models on the Massive Multitask Language Understanding benchmark. The script follows the repository's conventions and aims to achieve scores comparable to those reported on the Hugging Face page.

Changes

• Added examples/mmlu_eval.py with MMLU evaluation functionality
• Implemented dataset loading, prompt formatting, and accuracy calculation
• Added command-line interface for flexible configuration

Usage

python examples/mmlu_eval.py \
    --model_path=/path/to/model \
    --tokenizer_path=/path/to/tokenizer \
    --subject=all

Related Issues

Closes #36

[google-cla]

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

[Conchylicultor]

This seems to be AI generated. Closing

[Konikz]

Hi @Conchylicultor,

I believe the CLA check failed because my "keep my email address private" is enabled . I'll update my settings and reattempt the check.

[Konikz]

I tried adding my @users.noreply.github.com email to my CLA account to keep my email address private but since it's my uni email id, adding any address is restricted, is there a way around this ?