Skip to content

Conversation

@NathanHB
Copy link
Member

No description provided.

@HuggingFaceDocBuilderDev
Copy link
Collaborator

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Copilot finished reviewing on behalf of NathanHB November 12, 2025 15:55
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds a task dump functionality to the registry that exports all task names, metadata, and docstrings as JSON for better documentation. The changes also reduce the number of Global MMLU task variations.

  • Adds get_tasks_dump() method to the Registry class to serialize task configurations and metadata
  • Adds a new dump CLI command to output task information as JSON
  • Reduces Global MMLU task variations by removing CF and Hybrid formulations and limiting to "ALL" sensitivity label

Reviewed Changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File Description
src/lighteval/tasks/registry.py Adds instance variables to store initialization parameters and implements get_tasks_dump() method with docstring parsing and value serialization logic
src/lighteval/tasks/multilingual/tasks/global_mmlu.py Removes unused imports (CFFormulation, HybridFormulation) and reduces task variations
src/lighteval/main_tasks.py Adds new dump CLI command that uses the registry's task dump functionality to output JSON

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

NathanHB and others added 5 commits November 12, 2025 16:57
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Copilot finished reviewing on behalf of NathanHB November 12, 2025 16:11
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@NathanHB NathanHB merged commit b8ccd20 into main Nov 12, 2025
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants