Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Submission checker changes #1413

Merged
merged 5 commits into from
Jul 5, 2023
Merged

Submission checker changes #1413

merged 5 commits into from
Jul 5, 2023

Conversation

pgmpablo157321
Copy link
Contributor

Adress #1412

@github-actions
Copy link

github-actions bot commented Jun 22, 2023 •
edited
Loading

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅

@pgmpablo157321 pgmpablo157321 force-pushed the submission_checker_v3.1 branch 2 times, most recently from 8fe4b11 to ae7c9b1 Compare June 23, 2023 19:45
@pgmpablo157321 pgmpablo157321 marked this pull request as ready for review June 23, 2023 20:59
@pgmpablo157321 pgmpablo157321 requested a review from a team as a code owner June 23, 2023 20:59
@pgmpablo157321 pgmpablo157321 force-pushed the submission_checker_v3.1 branch 2 times, most recently from 9b374a3 to 67afeff Compare June 27, 2023 23:16
@pgmpablo157321
Copy link
Contributor Author

@arjunsuresh @ashwin Just to double check. Is removing this test here ok?

inference/tools/submission/submission_checker.py

Lines 3158 to 3162 in b22d0bc

if model in [
"gptj-99",
"gptj-99.9"
]:
test_list.remove("TEST01")

Does this comply with what we discussed in the WG meeting?

@arjunsuresh
Copy link
Contributor

@pgmpablo157321 TEST05 should be removed and TEST01 can stay.

Also, there was a PR merge in the power WG yesterday - so the checksums need to be updated again. Also, we need to add the final commit hash (from which submissions are valid) of the power-dev repository in the inference notes.

arjunsuresh
arjunsuresh reviewed Jun 28, 2023
View reviewed changes
tools/submission/submission_checker.py Outdated Show resolved Hide resolved
tools/submission/submission_checker.py Outdated Show resolved Hide resolved
arjunsuresh
arjunsuresh approved these changes Jun 28, 2023
View reviewed changes
Copy link
Contributor

@arjunsuresh arjunsuresh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @pgmpablo157321 for the changes. LGTM. GPT-3 latency can also be updated as 20s as decided in the taskforce meeting.

@arjunsuresh
Copy link
Contributor

@pgmpablo157321 @mrmhodak can you please merge this PR today? Then we can check the submission code for any issues over the weekend.

@mrmhodak mrmhodak merged commit d3385dd into master Jul 5, 2023
7 checks passed
@github-actions github-actions bot locked and limited conversation to collaborators Jul 5, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@szutenberg szutenberg szutenberg left review comments

@arjunsuresh arjunsuresh arjunsuresh approved these changes

@mrmhodak mrmhodak mrmhodak approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@pgmpablo157321 @arjunsuresh @mrmhodak @szutenberg