Late submissions restricted to only extraordinary circumstances

submission_rules.adoc

@@ -267,7 +267,10 @@ MLCommons provides a web UI [TODO: URL] for verifying scores contained in the ta
 ### Late Submissions
 
 #### Post-submission grace period (submission deadline + 60 minutes):
-MLPerf will allow submissions for up to 60 minutes after the published deadline without explanation or penalty. This grace period will be advertised as little as possible. The 60 minute limit will be strictly enforced.
+MLPerf will allow submissions for up to 60 minutes after the published deadline without explanation or penalty. The end of this grace period constitutes a hard deadline which will be strictly enforced.
+
+If a submitter cannot upload a valid submission tarball by the hard deadline due to some defect in the submission checker, a link to the tarball and a hash of the tarball must be sent to the submitters mailing list before the deadline. An issue must be raised on the relevant GitHub repository detailing the failure reason of the submission checker before the submission deadline. The burden is on the submitter to demonstrate that the submission could not be uploaded to via the web GUI because the checker was defective, and that the tarball passes the submission checker once the defect has been addressed. No other changes to the tarball other than those directly related to the submission checker fix will be allowed.
+
 
 #### Post-submission extension for extraordinary circumstances (submission deadline + 72 hours):
 If a submitter notifies the submission chair that their submission will be delayed due to force-majeure-type circumstances (e.g. blizzards, hurricanes, terrorism, etc.), the submission chair will delay sharing results for up to 72 hours to allow that submitter more time to make their submission. The extraordinary nature of the circumstances must be approved by the review committee at the first committee meeting or the submission will be disregarded. 
@@ -590,36 +593,10 @@ This section in progress [TODO].
 
 #### Inference
 
-Submitters must run the compliance tests for their closed and network divisions submissions to verify that their submission achieves a basic level of compliance with a subset of the MLPerf rules. If compliance testing identifies a potential issue with the submission, the onus is on the submitter to provide an adequate explanation to the results review committee.
+Submitters must run the compliance tests for their closed and network divisions submissions to verify that their submission achieves a basic level of compliance with a subset of the MLPerf rules. If a compliance test fails, the submission checker will not allow a submission to closed division. If the submission is instead made to open division, it may be moved to closed division upon an adequate explanation of the compliance test failure to the review committee.
 
 Refer to the documentation found under https://github.com/mlperf/inference/tree/master/compliance/nvidia
 
-### Submission Privacy
-
-Submitters may adopt the following procedure to ensure that their submission is shared only with others who have made valid submissions:
-
-By the submission deadline, the submitter will email the following items to submissions@mlcommons.org to show Proof of Work:
-
-- A URL to the location in public blob storage of an encrypted tarball containing the directory structure and files described in the <<directory-structure>> section above. Any storage mechanism that supports curl may be used.
-- A password to the encrypted tarball
-- A sha1sum of the tarball
-- The last two lines of submission_checker_log.txt as cursory evidence of a valid submission
-
-A simple script has been provided to facilitate this process: https://github.com/mlcommons/logging/blob/master/pack_submission.sh
-
-When the deadline is reached, repository access will be removed for everyone EXCEPT the following parties:
-
-- Chairs
-- Submitters who have submitted valid submissions
-
-For each submitted tarball, the results chair will then:
-
-- download the encrypted tarball
-- verify it matches the hash
-- decrypt the tarball
-- verify that it is a valid submission
-- add the submission to the repository
-- enable repository access for the submitter
 
 ## Review