From 97000f1b92561aace6ac426d4e4e3367199ad86e Mon Sep 17 00:00:00 2001 From: Ritika Borkar <54604832+nv-rborkar@users.noreply.github.com> Date: Thu, 13 Jan 2022 11:25:40 -0800 Subject: [PATCH 1/6] Update submission_rules.adoc --- submission_rules.adoc | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/submission_rules.adoc b/submission_rules.adoc index 892b303..4e4cc33 100644 --- a/submission_rules.adoc +++ b/submission_rules.adoc @@ -660,9 +660,9 @@ Results will be divided into categories based on the availability of the hardwar #### Available Systems -_Available_ cloud systems must (1) have available pricing (either publicly advertised or available by request), (2) have been rented by at least one third party, (3) have public evidence of availability (web page saying product is available, statement by company, etc), and (4) be “reasonably available” for rent by additional third parties by the submission date. +_Available_ cloud systems must (1) have available pricing (either publicly advertised or available by request), (2) have been rented by at least one third party, (3) have public evidence of availability (web page saying product is available and public statement by company), (4) be “reasonably available” for rent by additional third parties by the submission date, and (5) be accessible for 3rd party audit during MLPerf Review period. -An on-premise system is _Available_ if all of its components that substantially determine ML performance are _Available_ either individually or in aggregate (development boards that meet the substantially determine clause are allowed). An _Available_ component or system must (1) have available pricing (either publicly advertised or available by request), (2) have been shipped to at least one third party, (3) have public evidence of availability (web page saying product is available, statement by company, etc), and (4) be “reasonably available” for purchase by additional third parties by the submission date. In addition, submissions for on-premise systems must describe the system and its components in sufficient detail to enable third parties to build a similar system. +An on-premise system is _Available_ if all of its components that substantially determine ML performance are _Available_ either individually or in aggregate (development boards that meet the substantially determine clause are allowed). An _Available_ component or system must (1) have available pricing (either publicly advertised or available by request), (2) have been shipped to at least one third party, (3) have public evidence of availability (web page saying product is available and public statement by company), 4) be “reasonably available” for purchase by additional third parties by the submission date, and (5) be accessible for 3rd party audit during MLPerf Review period. In addition, submissions for on-premise systems must describe the system and its components in sufficient detail to enable third parties to build a similar system. In both cases, “reasonably available” means: From 2fdd558718edf919629f1ad521b005321f93805a Mon Sep 17 00:00:00 2001 From: Ritika Borkar <54604832+nv-rborkar@users.noreply.github.com> Date: Wed, 20 Jul 2022 08:56:44 -0700 Subject: [PATCH 2/6] Update submission_rules.adoc --- submission_rules.adoc | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/submission_rules.adoc b/submission_rules.adoc index 0286009..cb2b877 100644 --- a/submission_rules.adoc +++ b/submission_rules.adoc @@ -738,14 +738,13 @@ Results will be divided into categories based on the availability of the hardwar #### Available Systems -_Available_ cloud systems must (1) have available pricing (either publicly advertised or available by request), (2) have been rented by at least one third party, (3) have public evidence of availability (web page saying product is available and public statement by company), (4) be “reasonably available” for rent by additional third parties by the submission date, and (5) be accessible for 3rd party audit during MLPerf Review period. +_Available_ cloud systems must (1) have available pricing (either publicly advertised or available by request), (2) have been rented by at least one third party customer, (3) have public evidence of availability (web page saying product is available and public statement by company), (4) be “reasonably available” for rent by additional third parties by the submission date, and (5) be accessible to Results WG during MLPerf Review period to verify HW+SW availability (a scale smaller than submission scale is acceptable as long as the model compiles & runs) -An on-premise system is _Available_ if all of its components that substantially determine ML performance are _Available_ either individually or in aggregate (development boards that meet the substantially determine clause are allowed). An _Available_ component or system must (1) have available pricing (either publicly advertised or available by request), (2) have been shipped to at least one third party, (3) have public evidence of availability (web page saying product is available and public statement by company), 4) be “reasonably available” for purchase by additional third parties by the submission date, and (5) be accessible for 3rd party audit during MLPerf Review period. In addition, submissions for on-premise systems must describe the system and its components in sufficient detail to enable third parties to build a similar system. +An on-premise system is _Available_ if all of its components that substantially determine ML performance are _Available_ either individually or in aggregate (commercially available development boards that meet the substantially determine clause are allowed). An _Available_ component or system must (1) have available pricing (either publicly advertised or available by request), (2) have been shipped to at least one third party for production use, (3) have public evidence of availability (web page saying product is available and public statement by company), 4) be “reasonably available” for purchase by additional third parties by the submission date, and (5) be accessible to Results WG during MLPerf Review period to verify HW+SW availability (a scale smaller than submission scale is acceptable as long as the model compiles & runs). In addition, submissions for on-premise systems must describe the system and its components in sufficient detail to enable third parties to build a similar system. In both cases, “reasonably available” means: - 1. Supply and lead times are appropriate for system scale, i.e. on-demand and in quantity for the smallest systems and a few months and with limited supply for the largest systems. 2. Access to rent or purchase may be subject to conditions that are common to generally available products (such as financial qualifications, size of customer, support burden, export restrictions, etc.) but is not otherwise restricted (i.e. no “early access” approval requirements). From 2418bcac0657d8ae854c0cfd9193f88e999293c1 Mon Sep 17 00:00:00 2001 From: Ritika Borkar <54604832+nv-rborkar@users.noreply.github.com> Date: Wed, 20 Jul 2022 08:59:46 -0700 Subject: [PATCH 3/6] Update submission_rules.adoc --- submission_rules.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/submission_rules.adoc b/submission_rules.adoc index cb2b877..c2db2b9 100644 --- a/submission_rules.adoc +++ b/submission_rules.adoc @@ -738,7 +738,7 @@ Results will be divided into categories based on the availability of the hardwar #### Available Systems -_Available_ cloud systems must (1) have available pricing (either publicly advertised or available by request), (2) have been rented by at least one third party customer, (3) have public evidence of availability (web page saying product is available and public statement by company), (4) be “reasonably available” for rent by additional third parties by the submission date, and (5) be accessible to Results WG during MLPerf Review period to verify HW+SW availability (a scale smaller than submission scale is acceptable as long as the model compiles & runs) +_Available_ cloud systems must (1) have available pricing (either publicly advertised or available by request), (2) have been rented by at least one third party, (3) have public evidence of availability (web page saying product is available and public statement by company), (4) be “reasonably available” for rent by additional third parties by the submission date, and (5) be accessible to Results WG during MLPerf Review period to verify HW+SW availability (a scale smaller than submission scale is acceptable as long as the model compiles & runs) An on-premise system is _Available_ if all of its components that substantially determine ML performance are _Available_ either individually or in aggregate (commercially available development boards that meet the substantially determine clause are allowed). An _Available_ component or system must (1) have available pricing (either publicly advertised or available by request), (2) have been shipped to at least one third party for production use, (3) have public evidence of availability (web page saying product is available and public statement by company), 4) be “reasonably available” for purchase by additional third parties by the submission date, and (5) be accessible to Results WG during MLPerf Review period to verify HW+SW availability (a scale smaller than submission scale is acceptable as long as the model compiles & runs). In addition, submissions for on-premise systems must describe the system and its components in sufficient detail to enable third parties to build a similar system. From 97b111718dce1a06916dc8c48132da631a838a3b Mon Sep 17 00:00:00 2001 From: Ritika Borkar <54604832+nv-rborkar@users.noreply.github.com> Date: Thu, 28 Jul 2022 09:03:12 -0700 Subject: [PATCH 4/6] Update submission_rules.adoc --- submission_rules.adoc | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/submission_rules.adoc b/submission_rules.adoc index c2db2b9..8ca0549 100644 --- a/submission_rules.adoc +++ b/submission_rules.adoc @@ -738,9 +738,9 @@ Results will be divided into categories based on the availability of the hardwar #### Available Systems -_Available_ cloud systems must (1) have available pricing (either publicly advertised or available by request), (2) have been rented by at least one third party, (3) have public evidence of availability (web page saying product is available and public statement by company), (4) be “reasonably available” for rent by additional third parties by the submission date, and (5) be accessible to Results WG during MLPerf Review period to verify HW+SW availability (a scale smaller than submission scale is acceptable as long as the model compiles & runs) +_Available_ cloud systems must (1) have available pricing (either publicly advertised or available by request), (2) have been rented by at least one third party customer for use (eg: ML or compute use), (3) have public evidence of availability (web page saying product is available and public statement by company), (4) be “reasonably available” for rent by additional third party customers by the submission date, and (5) be accessible to Results WG during MLPerf Review period to verify HW+SW availability (a scale smaller than submission scale is acceptable as long as the model compiles & runs) -An on-premise system is _Available_ if all of its components that substantially determine ML performance are _Available_ either individually or in aggregate (commercially available development boards that meet the substantially determine clause are allowed). An _Available_ component or system must (1) have available pricing (either publicly advertised or available by request), (2) have been shipped to at least one third party for production use, (3) have public evidence of availability (web page saying product is available and public statement by company), 4) be “reasonably available” for purchase by additional third parties by the submission date, and (5) be accessible to Results WG during MLPerf Review period to verify HW+SW availability (a scale smaller than submission scale is acceptable as long as the model compiles & runs). In addition, submissions for on-premise systems must describe the system and its components in sufficient detail to enable third parties to build a similar system. +An on-premise system is _Available_ if all of its components that substantially determine ML performance are _Available_ either individually or in aggregate (commercially available development boards that meet the substantially determine clause are allowed). An _Available_ component or system must (1) have available pricing (either publicly advertised or available by request), (2) have been shipped to at least one third party customer for use (eg: ML or compute use), (3) have public evidence of availability (web page saying product is available and public statement by company), 4) be “reasonably available” for purchase by additional third parties by the submission date, and (5) be accessible to Results WG during MLPerf Review period to verify HW+SW availability (a scale smaller than submission scale is acceptable as long as the model compiles & runs). In addition, submissions for on-premise systems must describe the system and its components in sufficient detail to enable third parties to build a similar system. In both cases, “reasonably available” means: From f7789c48a4933f86e896a8f8aa2e43725c4b4859 Mon Sep 17 00:00:00 2001 From: Ritika Borkar <54604832+nv-rborkar@users.noreply.github.com> Date: Thu, 28 Jul 2022 10:34:04 -0700 Subject: [PATCH 5/6] Update submission_rules.adoc --- submission_rules.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/submission_rules.adoc b/submission_rules.adoc index 8ca0549..a7379b9 100644 --- a/submission_rules.adoc +++ b/submission_rules.adoc @@ -740,7 +740,7 @@ Results will be divided into categories based on the availability of the hardwar _Available_ cloud systems must (1) have available pricing (either publicly advertised or available by request), (2) have been rented by at least one third party customer for use (eg: ML or compute use), (3) have public evidence of availability (web page saying product is available and public statement by company), (4) be “reasonably available” for rent by additional third party customers by the submission date, and (5) be accessible to Results WG during MLPerf Review period to verify HW+SW availability (a scale smaller than submission scale is acceptable as long as the model compiles & runs) -An on-premise system is _Available_ if all of its components that substantially determine ML performance are _Available_ either individually or in aggregate (commercially available development boards that meet the substantially determine clause are allowed). An _Available_ component or system must (1) have available pricing (either publicly advertised or available by request), (2) have been shipped to at least one third party customer for use (eg: ML or compute use), (3) have public evidence of availability (web page saying product is available and public statement by company), 4) be “reasonably available” for purchase by additional third parties by the submission date, and (5) be accessible to Results WG during MLPerf Review period to verify HW+SW availability (a scale smaller than submission scale is acceptable as long as the model compiles & runs). In addition, submissions for on-premise systems must describe the system and its components in sufficient detail to enable third parties to build a similar system. +An on-premise system is _Available_ if all of its components that substantially determine ML performance are _Available_ either individually or in aggregate (commercially available development boards that meet the substantially determine clause are allowed). An _Available_ component or system must (1) have available pricing (either publicly advertised or available by request), (2) have been shipped to at least one third party customer for use (eg: ML or compute use; System vendor partners do not qualify as customers), (3) have public evidence of availability (web page saying product is available and public statement by company), 4) be “reasonably available” for purchase by additional third parties by the submission date, and (5) be accessible to Results WG during MLPerf Review period to verify HW+SW availability (a scale smaller than submission scale is acceptable as long as the model compiles & runs). In addition, submissions for on-premise systems must describe the system and its components in sufficient detail to enable third parties to build a similar system. In both cases, “reasonably available” means: From 0fd6a8c8e077d53384df0d2eecb384e8f2e58dae Mon Sep 17 00:00:00 2001 From: Ritika Borkar <54604832+nv-rborkar@users.noreply.github.com> Date: Thu, 4 Aug 2022 08:34:54 -0700 Subject: [PATCH 6/6] Update submission_rules.adoc --- submission_rules.adoc | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/submission_rules.adoc b/submission_rules.adoc index a7379b9..e4a5329 100644 --- a/submission_rules.adoc +++ b/submission_rules.adoc @@ -738,9 +738,9 @@ Results will be divided into categories based on the availability of the hardwar #### Available Systems -_Available_ cloud systems must (1) have available pricing (either publicly advertised or available by request), (2) have been rented by at least one third party customer for use (eg: ML or compute use), (3) have public evidence of availability (web page saying product is available and public statement by company), (4) be “reasonably available” for rent by additional third party customers by the submission date, and (5) be accessible to Results WG during MLPerf Review period to verify HW+SW availability (a scale smaller than submission scale is acceptable as long as the model compiles & runs) +_Available_ cloud systems must (1) have available pricing (either publicly advertised or available by request), (2) have been rented by at least one third party customer for use (eg: ML or compute use), (3) have public evidence of availability (web page saying product is available and public statement by company), and (4) be “reasonably available” for rent by additional third party customers by the submission date. -An on-premise system is _Available_ if all of its components that substantially determine ML performance are _Available_ either individually or in aggregate (commercially available development boards that meet the substantially determine clause are allowed). An _Available_ component or system must (1) have available pricing (either publicly advertised or available by request), (2) have been shipped to at least one third party customer for use (eg: ML or compute use; System vendor partners do not qualify as customers), (3) have public evidence of availability (web page saying product is available and public statement by company), 4) be “reasonably available” for purchase by additional third parties by the submission date, and (5) be accessible to Results WG during MLPerf Review period to verify HW+SW availability (a scale smaller than submission scale is acceptable as long as the model compiles & runs). In addition, submissions for on-premise systems must describe the system and its components in sufficient detail to enable third parties to build a similar system. +An on-premise system is _Available_ if all of its components that substantially determine ML performance are _Available_ either individually or in aggregate (commercially available development boards that meet the substantially determine clause are allowed). An _Available_ component or system must (1) have available pricing (either publicly advertised or available by request), (2) have been shipped to at least one third party customer for use (eg: ML or compute use; System vendor partners do not qualify as customers), (3) have public evidence of availability (web page saying product is available and public statement by company), and (4) be “reasonably available” for purchase by additional third parties by the submission date. In addition, submissions for on-premise systems must describe the system and its components in sufficient detail to enable third parties to build a similar system. In both cases, “reasonably available” means: