Permalink
Find file Copy path
a167e6a Dec 13, 2018
1 contributor

Users who have contributed to this file

63 lines (33 sloc) 6.83 KB

MLPerf Terms of Use

These are the Terms of Use for MLPerf, including benchmark results and the MLPerf.org website.

MLPerf name and logo are trademarks; use must conform to this document

The MLPerf name and logo are trademarks. Any use of the MLPerf trademark must conform to these terms of use. The MLPerf organization reserves the right to solely determine if a use of its name or logo is acceptable. If you use a benchmark result in a manner referencing an MLPerf trademark, you must conform to the following guidelines.

The nature and source of cited MLPerf results must be clear

All results used in conjunction with the MLPerf trademark must include the following: In primary citation: submitting organization, benchmark name*, and system under test**. For example:

SmartAI Corp achieved a score of 0.6 on the MLPerf Image Classification benchmark using a 10 SmartChip cluster.

* For Closed Division benchmarks, the model name may be used instead of the benchmark name, e.g. "SSD" instead of "Object Detection (light-weight)".

** System description must be consistent with the description in the official MLPerf results. Furthermore, chip count must be included if another submitter’s score for the same benchmark is listed in the document.

In primary citation or footnote: benchmark version and division***, ML Framework and version, accelerator library software and version (as applicable), link to MLPerf submission, date and source of retrieval, MLPerf result ID (major-version.minor-version.entry.benchmark), and clear reference to MLPerf trademark. For example:

[1] MLPerf v0.5 Training Closed; system employed ML Framework v4.1 with the MLNN v7.4 library. Retrieved from www.mlperf.org 21 December 2018, entry 0.5.12.2. MLPerf name and logo are trademarks. See www.mlperf.org for more information.

*** These data items must be included in the primary citation if results from different benchmark versions or divisions are being compared.

Official results must be clearly distinguished from unofficial results

You may cite either official results obtained from the MLPerf results page or unofficial results measured independently. If you cite an unofficial result you must clearly specify that the result is “Unverified” in text and clearly state “Result not verified by MLPerf” in a footnote. The result must comply with the letter and spirit of the relevant MLPerf rules. For example:

SmartAI Corp announced an estimated score of 0.3 on the MLPerf v0.5 Training Closed Division - Image Classification benchmark using a cluster of 20 SmartChips running MLFramework v4.1 [1].

[1] Result not verified by MLPerf. MLPerf name and logo are trademarks. See www.mlperf.org for more information.

MLPerf allows but does not endorse combining results of benchmarks

Users may see fit to combine or aggregate results from multiple MLPerf benchmark tests and/or other 3rd party results. If publicly disclosed, these composite results must cite MLPerf as required above and clearly describe the method of combination. However the composite result is not sanctioned by MLPerf and may not be represented as an official MLPerf result or score.

Comparisons based on secondary or derived metrics must be explicit

Each MLPerf benchmark has a primary metric, for instance time-to-train for Training Image Classification. Any comparison based on different or derived metric such as power, cost, model size/architecture, accuracy, etc. must make the basis for comparison clear in the text and in a footnote. Secondary and derived metrics must not be presented as official MLPerf metrics.

Prestigious Research University has created a new neural network model called MagicEightBall that is 100% accurate for Top-1 image classification on the MLPerf v0.5 Training Open Division Image Classification benchmark using a cluster of 10 SmartChips running MLFramework v4.1 [1].

[1] Accuracy is not the primary metric of MLPerf. MLPerf name and logo are registered trademarks. See www.mlperf.org for more information.

Comparisons between major revisions of an MLPerf benchmark are not allowed

Whether comparing official results or estimates, comparisons must be made between results of compatible revisions of an MLPerf benchmark. Compatible revisions are determined by the MLPerf organization and are described in each benchmark’s documentation.

The MLPerf website is hosted by Google; the following terms apply

In addition to the Google Terms of Service and Privacy Policy, the following terms will apply to your use of the MLPerf website (the “Website”), service (the “Service), and software (the “Software”). If you do not accept these terms and conditions in full, you must terminate the use of the Website, Service, and Software immediately. THE SERVICE AND SOFTWARE ARE PROVIDED TO YOU "AS IS" WITHOUT ANY WARRANTY OR IMPLIED WARRANTY. NEITHER MLPERF NOR ITS PARENT, SUBSIDIARIES, OR AFFILIATES (“MLPERF”) TAKE ANY RESPONSIBILITY FOR ANY DIFFICULTIES YOU MAY ENCOUNTER WITH THE SERVICE OR SOFTWARE. MLPERF DOES NOT WARRANT THAT THE FUNCTIONS CONTAINED IN THE SOFTWARE OR OTHER PRODUCTS WILL MEET YOUR REQUIREMENTS, OR THAT THE OPERATION OF THE SOFTWARE WILL BE UNINTERRUPTED OR ERROR-FREE, OR THAT DEFECTS IN THE SOFTWARE OR OTHER PRODUCTS WILL BE CORRECTED. NO ORAL OR WRITTEN INFORMATION, BENCHMARKS, BENCHMARK RESULTS, OR ADVICE GIVEN BY MLPERF (THE “MLPERF CONTENT”) SHALL CREATE ANY WARRANTY.

MLPERF IS NOT RESPONSIBLE FOR ANY PRODUCTS YOU MAY CHOOSE TO PURCHASE OR CHOOSE NOT TO PURCHASE AS A RESULT OF THE MLPERF CONTENT. MLPERF DISCLAIMS ANY AND ALL LOSS OR LIABILITY RELATED TO YOUR USE OF THE WEBSITE, THE MLPERF CONTENT GIVEN TO YOU ON THE WEBSITE OR SOFTWARE FROM THE WEBSITE, OR THIS AGREEMENT.

MLPerf is not liable for, among other things, any loss of data, hardware or software, or any liability resulting from: access delays; access interruptions; viruses; hackers; crackers; data non-delivery or mis-delivery; negligent acts, grossly negligent acts, or omissions by MLPerf; errors in any information, goods, or documents obtained due to the MLPerf Content; or force majeure, or any damages whatsoever resulting from loss of use, data or profits, whether in an action of contract, negligence or other action, arising out of or in connection with the use or performance of the Website, the MLPerf Content, the Service or the Software.

MLPerf makes no representations whatsoever about any other website that you may access through the Website. MLPerf has no control over the content or claims of websites outside the MLPerf domain, and does not endorse or accept any responsibility for the content of such websites.

MLPERF, ITS REPRESENTATIVES, LICENSORS, AND PARTNERS WILL NOT BE LIABLE FOR ANY DAMAGES (INCLUDING DIRECT, INDIRECT, INCIDENTAL, CONSEQUENTIAL, SPECIAL OR PUNITIVE).