Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

mmlu branch run() complete #19

Merged
merged 3 commits into from
Jun 26, 2024
Merged

mmlu branch run() complete #19

merged 3 commits into from
Jun 26, 2024

Conversation

alimaredia
Copy link
Contributor

No description provided.

danmcp
danmcp requested changes Jun 26, 2024
View reviewed changes
src/instructlab/eval/mmlu.py Outdated Show resolved Hide resolved
src/instructlab/eval/mmlu.py Outdated Show resolved Hide resolved
alimaredia and others added 2 commits June 26, 2024 14:01
@alimaredia @nathan-weinberg
mmlu branch run() complete
f3bbadd 
Signed-off-by: Ali Maredia <amaredia@redhat.com>
@nathan-weinberg
Address review comments
50a4a36 
Signed-off-by: Nathan Weinberg <nweinber@redhat.com>
@nathan-weinberg
Fix linter
fc1aa98 
Signed-off-by: Nathan Weinberg <nweinber@redhat.com>
JamesKunstle
JamesKunstle approved these changes Jun 26, 2024
View reviewed changes
Copy link
Contributor

@JamesKunstle JamesKunstle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

alinaryan
alinaryan approved these changes Jun 26, 2024
View reviewed changes
Copy link
Member

@alinaryan alinaryan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This LGTM. We should run the old version on a particular checkpoint, record the number, and then run the new version to make sure everything is the same, like Kai suggested. If it differs, it can be fixed in a followup PR, but let's get this in for now.

danmcp
danmcp approved these changes Jun 26, 2024
View reviewed changes
Copy link
Member

@danmcp danmcp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving but noting the future cleanup.

@nathan-weinberg
Copy link
Member

Approving but noting the future cleanup.

I'll open an issue for tracking

@nathan-weinberg nathan-weinberg merged commit 8494a51 into instructlab:main Jun 26, 2024
6 checks passed
@nathan-weinberg nathan-weinberg mentioned this pull request Jun 26, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@nathan-weinberg nathan-weinberg nathan-weinberg left review comments

@danmcp danmcp danmcp approved these changes

@JamesKunstle JamesKunstle JamesKunstle approved these changes

@alinaryan alinaryan alinaryan approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@alimaredia @nathan-weinberg @danmcp @JamesKunstle @alinaryan