Skip to content

Conversation

@fsschneider
Copy link
Contributor

@fsschneider fsschneider commented Nov 16, 2023

This PR adds the Competition Rules, moves what was previously called the rules to a technical documentation page, adds the Call for Submissions, and makes a few rules changes as discussed in the WG meetings.

@georgedahl I want to highlight the following changes in particular, that could use your attention:

  • The FAQs are currently part of the "(Technical) Documentation" doc. Should it rather be a separate file? @priyakasimbeg?
    • I added the question "Can I submit multiple times to the benchmark competition?" with the answer that was originally in the "Submission Process Rules" document. Is this answer okay?
    • How can we better phrase the question and answer about "submitting existing algorithms"?
  • I removed mentions of the spirit jury in the "Technical Documentation" doc, see this commit. Any edits you'd like to see (in particular to this section)
  • I clarified that we do tuning also for the held-out workloads, see this commit. Is the text okay?
  • I added also a Submission (Google) Form (similar to the Intent Registration Form) to the Competition Rules. It is currently just a placeholder, saying that the submission portal is currently not open.
  • I entirely removed the "Submission Process Rules" as most of it is contained within the "Competition Rules" now. A few notable exceptions are noted below. Is that okay with you that those got removed?
    • There is no statement anymore that members of the awards committee are ineligible for prize money.
    • There is no deadline for a version freeze anymore. Instead, there is now this section on "Version freeze" in the "Technical Documentation" as well as the first sentence of the "Cancellation and Modification" Section in the "Competition Rules".

To-dos before merge

  • Specify the threshold submissions (for both rulesets) whose score needs to be exceeded for prize money.
  • Get approvable by Peter and David
  • Double-check with the lawyers
  • Update deadline dates

@github-actions
Copy link

github-actions bot commented Nov 16, 2023

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅

priyakasimbeg
priyakasimbeg previously approved these changes Nov 16, 2023
Copy link
Contributor

@priyakasimbeg priyakasimbeg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.
I think we can leave the FAQs in the DOCUMENTATION or README because it will be more likely people will notice the section. Alternatively if you think it is too large we can make a separate file and link it from the DOCUMENTATION or README? Either way is fine with me.

georgedahl
georgedahl previously approved these changes Nov 18, 2023
Copy link
Contributor

@georgedahl georgedahl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add a review comment to any part you think I still need to look at?

@fsschneider fsschneider dismissed stale reviews from georgedahl and priyakasimbeg via fb811f6 November 23, 2023 13:38
@fsschneider fsschneider requested review from georgedahl and removed request for TheKanter and petermattson November 23, 2023 13:42
georgedahl
georgedahl previously approved these changes Nov 27, 2023
@fsschneider fsschneider merged commit d2bbc46 into mlcommons:dev Nov 28, 2023
@github-actions github-actions bot locked and limited conversation to collaborators Nov 28, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants