Skip to content

feat: add AIME-2025 eval dataset.#777

Merged
terrykong merged 2 commits intoNVIDIA-NeMo:mainfrom
xxman-google:xx/aime2025
Jul 30, 2025
Merged

feat: add AIME-2025 eval dataset.#777
terrykong merged 2 commits intoNVIDIA-NeMo:mainfrom
xxman-google:xx/aime2025

Conversation

@xxman-google
Copy link
Contributor

What does this PR do ?

Add AIME-2025 eval dataset.

Issues

List issues that this PR closes (syntax): N/A

Usage

  • Run evaluation on AIME-2025
uv run examples/run_eval.py --config examples/configs/evals/aime2025.yaml cluster.gpus_per_node=8

Before your PR is "Ready for review"

Pre checks:

  • Make sure you read and followed Contributor guidelines
  • Did you write any new necessary tests?
  • Did you run the unit tests and functional tests locally? Visit our Testing Guide for how to run tests
  • Did you add or update any necessary documentation? Visit our Document Development Guide for how to write, build and test the docs.

Additional Information

  • ...

@github-actions github-actions bot added the documentation Improvements or additions to documentation label Jul 29, 2025
@xxman-google xxman-google changed the title add AIME-2025 eval dataset. feat: add AIME-2025 eval dataset. Jul 29, 2025
@terrykong terrykong requested a review from yuki-97 July 29, 2025 01:05
Copy link
Contributor

@yuki-97 yuki-97 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @xxman-google for improving our evaluation! Left some comments.

Signed-off-by: Xuehan <xxman@google.com>
Copy link
Contributor Author

@xxman-google xxman-google left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

comments adressed. @YUki-666 . PTAL again.

yuki-97
yuki-97 previously approved these changes Jul 30, 2025
Copy link
Contributor

@yuki-97 yuki-97 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @xxman-google , LGTM! Approved.

@terrykong Can you take a review and enqueue it if it looks good to you?

@yuki-97 yuki-97 requested a review from terrykong July 30, 2025 03:17
Signed-off-by: Xuehan <xxman@google.com>
@xxman-google
Copy link
Contributor Author

@YUki-666 can you approve again? I added a new file to pyrefly.toml to avoid one CI error.

@terrykong terrykong enabled auto-merge July 30, 2025 05:00
@terrykong terrykong added this pull request to the merge queue Jul 30, 2025
Merged via the queue into NVIDIA-NeMo:main with commit dd5dc68 Jul 30, 2025
15 checks passed
@xxman-google xxman-google deleted the xx/aime2025 branch July 30, 2025 14:07
xxman-google added a commit to xxman-google/NeMo-RL that referenced this pull request Jul 30, 2025
Signed-off-by: Xuehan <xxman@google.com>
tpoisonooo pushed a commit to tpoisonooo/RL that referenced this pull request Aug 4, 2025
Signed-off-by: Xuehan <xxman@google.com>
Signed-off-by: tpoisonooo <khj.application@aliyun.com>
FannYYW pushed a commit to xxman-google/NeMo-RL that referenced this pull request Aug 5, 2025
Signed-off-by: Xuehan <xxman@google.com>
FannYYW pushed a commit to xxman-google/NeMo-RL that referenced this pull request Aug 5, 2025
Signed-off-by: Xuehan <xxman@google.com>
soodoshll pushed a commit to soodoshll/RL that referenced this pull request Aug 13, 2025
Signed-off-by: Xuehan <xxman@google.com>
Signed-off-by: Qidong Su <qidongs@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

community-request documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants