Link to mlcommons@cm4mlops (#1206)

mlcommons · Apr 16, 2024 · f22a3fe · f22a3fe
2 parents c24d9b4 + b8fb1e7
commit f22a3fe
Show file tree

Hide file tree

Showing 264 changed files with 1,076 additions and 98 deletions.
diff --git a/.github/workflows/test-qaic-compute-sdk-build.yml b/.github/workflows/test-qaic-compute-sdk-build.yml
@@ -27,7 +27,7 @@ jobs:
     - name: Install dependencies
       run: |
         python3 -m pip install cmind
-        cm pull repo mlcommons@ck
+        cm pull repo mlcommons@cm4mlops
         cm run script --tags=get,sys-utils-cm --quiet
   
     - name: Test QAIC Compute SDK for compilation

diff --git a/.github/workflows/test-qaic-software-kit.yml b/.github/workflows/test-qaic-software-kit.yml
@@ -32,7 +32,7 @@ jobs:
     - name: Install dependencies
       run: |
         python3 -m pip install cmind
-        cm pull repo mlcommons@ck
+        cm pull repo mlcommons@cm4mlops
         cm run script --tags=get,sys-utils-cm --quiet
   
     - name: Test Software Kit for compilation on Ubuntu 20.04

diff --git a/README.md b/README.md
@@ -57,7 +57,7 @@ using a few CM commands:
 
 pip install cmind -U
 
-cm pull repo mlcommons@ck
+cm pull repo mlcommons@cm4mlops
 
 cm rm cache -f
 
@@ -152,10 +152,10 @@ to generate CM commands that will prepare and run MLPerf benchmarks and AI appli
 ```bash
 pip install cmind -U
 
-cm pull repo mlcommons@ck
+cm pull repo mlcommons@cm4mlops
 
-cm checkout repo mlcommons@ck --branch=dev
-cm checkout repo mlcommons@ck --branch=master
+cm checkout repo mlcommons@cm4mlops --branch=dev
+cm checkout repo mlcommons@cm4mlops --branch=master
 
 cm rm cache -f
 
@@ -205,7 +205,7 @@ cmr "get generic-python-lib _package.torchvision" --version=0.16.2
 cmr "python app image-classification torch" --input=computer_mouse.jpg
 
 
-cm rm repo mlcommons@ck
+cm rm repo mlcommons@cm4mlops
 cm pull repo --url=https://zenodo.org/records/10787459/files/cm-mlops-repo-20240306.zip
 
 cmr "install llvm prebuilt" --version=17.0.6

diff --git a/cm-mlops/README.md b/cm-mlops/README.md
@@ -1,30 +1 @@
-# CM repository with cross-platform scripts for MLOps and DevOps
-
-[![License](https://img.shields.io/badge/License-Apache%202.0-green)](https://github.com/mlcommons/ck/tree/master/cm)
-[![Supported by CM](https://img.shields.io/badge/Supported_by-MLCommons%20CM-blue)](https://github.com/mlcommons/ck).
-
-This repository contains automation recipes to run native MLOps and DevOps scripts and tools via a simple, common and human-readable
-[CM interface](https://github.com/mlcommons/ck) on any platform with any software stack.
-
-All СM scripts have a simple Python API, extensible JSON/YAML meta description
-and unifed input/output to make them reusable in different projects either individually 
-or by chaining them together into powerful, efficient and portable automation workflows, 
-applications and web services adaptable to continuously changing models, data sets, software and hardware
-(see [CM workflow/pipeline for MLPerf inference](script/app-mlperf-inference/_cm.yaml) as example).
-
-These automation recipes are being developed and maintained by the [MLCommons Task Force on Automation and Reproducibility](../docs/taskforce.md)
-with [great contributions](../CONTRIBUTING.md) from the community and important feedback from Google, AMD, Neural Magic, OctoML, Nvidia, Qualcomm, Dell, HPE, 
-Red Hat, Intel, TTA, One Stop Systems, ACM and other organizations.
-
-### Catalog
-
-See the automatically generated catalog at [GitHub](https://access.cknowledge.org/playground/?action=scripts)
-or via the [Collective Knowledge playground](https://access.cknowledge.org/playground/?action=components).
-
-### License
-
-[Apache 2.0](LICENSE.md)
-
-### Copyright
-
-2022-2024 [MLCommons](https://mlcommons.org)
+**This repository is archived. Please use [mlcommons@cm4mlops](https://github.com/mlcommons/cm4mlops/tree/dev) instead!**
diff --git a/cm-mlops/automation/script/module_misc.py b/cm-mlops/automation/script/module_misc.py
@@ -261,6 +261,10 @@ def doc(i):
 #                            '{{CM_README_TOC}}',
 #                            '</details>',
 #                            '',
+                            '**Note that this script is archived and moved [here](https://github.com/mlcommons/cm4mlops/tree/main/script/{}).**'.format(meta['alias']),
+                            '',
+                            '',
+                            '',
                             'Automatically generated README for this automation recipe: **{}**'.format(meta['alias']), 
                             ]
 

diff --git a/cm-mlops/script/activate-python-venv/README.md b/cm-mlops/script/activate-python-venv/README.md
@@ -1,3 +1,7 @@
+**Note that this script is archived and moved [here](https://github.com/mlcommons/cm4mlops/tree/main/script/activate-python-venv).**
+
+
+
 Automatically generated README for this automation recipe: **activate-python-venv**
 
 Category: **Python automation**

diff --git a/cm-mlops/script/add-custom-nvidia-system/README.md b/cm-mlops/script/add-custom-nvidia-system/README.md
@@ -1,3 +1,7 @@
+**Note that this script is archived and moved [here](https://github.com/mlcommons/cm4mlops/tree/main/script/add-custom-nvidia-system).**
+
+
+
 Automatically generated README for this automation recipe: **add-custom-nvidia-system**
 
 Category: **MLPerf benchmark support**

diff --git a/cm-mlops/script/app-image-classification-onnx-py/README.md b/cm-mlops/script/app-image-classification-onnx-py/README.md
@@ -1,3 +1,7 @@
+**Note that this script is archived and moved [here](https://github.com/mlcommons/cm4mlops/tree/main/script/app-image-classification-onnx-py).**
+
+
+
 Automatically generated README for this automation recipe: **app-image-classification-onnx-py**
 
 Category: **Modular AI/ML application pipeline**

diff --git a/cm-mlops/script/app-image-classification-tf-onnx-cpp/README.md b/cm-mlops/script/app-image-classification-tf-onnx-cpp/README.md
@@ -1,3 +1,7 @@
+**Note that this script is archived and moved [here](https://github.com/mlcommons/cm4mlops/tree/main/script/app-image-classification-tf-onnx-cpp).**
+
+
+
 Automatically generated README for this automation recipe: **app-image-classification-tf-onnx-cpp**
 
 Category: **Modular AI/ML application pipeline**

diff --git a/cm-mlops/script/app-image-classification-torch-py/README.md b/cm-mlops/script/app-image-classification-torch-py/README.md
@@ -1,3 +1,7 @@
+**Note that this script is archived and moved [here](https://github.com/mlcommons/cm4mlops/tree/main/script/app-image-classification-torch-py).**
+
+
+
 Automatically generated README for this automation recipe: **app-image-classification-torch-py**
 
 Category: **Modular AI/ML application pipeline**

diff --git a/cm-mlops/script/app-image-classification-tvm-onnx-py/README.md b/cm-mlops/script/app-image-classification-tvm-onnx-py/README.md
@@ -1,3 +1,7 @@
+**Note that this script is archived and moved [here](https://github.com/mlcommons/cm4mlops/tree/main/script/app-image-classification-tvm-onnx-py).**
+
+
+
 Automatically generated README for this automation recipe: **app-image-classification-tvm-onnx-py**
 
 Category: **Modular AI/ML application pipeline**

diff --git a/cm-mlops/script/app-image-corner-detection/README.md b/cm-mlops/script/app-image-corner-detection/README.md
@@ -1,3 +1,7 @@
+**Note that this script is archived and moved [here](https://github.com/mlcommons/cm4mlops/tree/main/script/app-image-corner-detection).**
+
+
+
 Automatically generated README for this automation recipe: **app-image-corner-detection**
 
 Category: **Modular application pipeline**

diff --git a/cm-mlops/script/app-loadgen-generic-python/README.md b/cm-mlops/script/app-loadgen-generic-python/README.md
@@ -1,3 +1,7 @@
+**Note that this script is archived and moved [here](https://github.com/mlcommons/cm4mlops/tree/main/script/app-loadgen-generic-python).**
+
+
+
 Automatically generated README for this automation recipe: **app-loadgen-generic-python**
 
 Category: **Modular MLPerf inference benchmark pipeline**
@@ -127,7 +131,7 @@ ___
       - Workflow:
         1. ***Read "deps" on other CM scripts***
            * get,ml-model,cmc
-             - *Warning: no scripts found*
+             - CM script: [get-ml-model-abtf-ssd-pytorch](https://github.com/mlcommons/cm4abtf/tree/master/script/get-ml-model-abtf-ssd-pytorch)
     * `_custom,huggingface`
       - Workflow:
         1. ***Read "deps" on other CM scripts***

diff --git a/cm-mlops/script/app-mlperf-inference-ctuning-cpp-tflite/README.md b/cm-mlops/script/app-mlperf-inference-ctuning-cpp-tflite/README.md
@@ -1,3 +1,7 @@
+**Note that this script is archived and moved [here](https://github.com/mlcommons/cm4mlops/tree/main/script/app-mlperf-inference-ctuning-cpp-tflite).**
+
+
+
 Automatically generated README for this automation recipe: **app-mlperf-inference-ctuning-cpp-tflite**
 
 Category: **Modular MLPerf inference benchmark pipeline**

diff --git a/cm-mlops/script/app-mlperf-inference-dummy/README.md b/cm-mlops/script/app-mlperf-inference-dummy/README.md
@@ -1,3 +1,7 @@
+**Note that this script is archived and moved [here](https://github.com/mlcommons/cm4mlops/tree/main/script/app-mlperf-inference-dummy).**
+
+
+
 Automatically generated README for this automation recipe: **app-mlperf-inference-dummy**
 
 Category: **Modular MLPerf benchmarks**

diff --git a/cm-mlops/script/app-mlperf-inference-intel/README.md b/cm-mlops/script/app-mlperf-inference-intel/README.md
@@ -1,3 +1,7 @@
+**Note that this script is archived and moved [here](https://github.com/mlcommons/cm4mlops/tree/main/script/app-mlperf-inference-intel).**
+
+
+
 Automatically generated README for this automation recipe: **app-mlperf-inference-intel**
 
 Category: **Modular MLPerf benchmarks**

diff --git a/cm-mlops/script/app-mlperf-inference-mlcommons-cpp/README.md b/cm-mlops/script/app-mlperf-inference-mlcommons-cpp/README.md
@@ -1,3 +1,7 @@
+**Note that this script is archived and moved [here](https://github.com/mlcommons/cm4mlops/tree/main/script/app-mlperf-inference-mlcommons-cpp).**
+
+
+
 Automatically generated README for this automation recipe: **app-mlperf-inference-mlcommons-cpp**
 
 Category: **Modular MLPerf inference benchmark pipeline**

diff --git a/cm-mlops/script/app-mlperf-inference-mlcommons-python/README.md b/cm-mlops/script/app-mlperf-inference-mlcommons-python/README.md
@@ -1,3 +1,7 @@
+**Note that this script is archived and moved [here](https://github.com/mlcommons/cm4mlops/tree/main/script/app-mlperf-inference-mlcommons-python).**
+
+
+
 Automatically generated README for this automation recipe: **app-mlperf-inference-mlcommons-python**
 
 Category: **Modular MLPerf inference benchmark pipeline**

diff --git a/cm-mlops/script/app-mlperf-inference-nvidia/README.md b/cm-mlops/script/app-mlperf-inference-nvidia/README.md
@@ -1,3 +1,7 @@
+**Note that this script is archived and moved [here](https://github.com/mlcommons/cm4mlops/tree/main/script/app-mlperf-inference-nvidia).**
+
+
+
 Automatically generated README for this automation recipe: **app-mlperf-inference-nvidia**
 
 Category: **Reproduce MLPerf benchmarks**

diff --git a/cm-mlops/script/app-mlperf-inference-qualcomm/README.md b/cm-mlops/script/app-mlperf-inference-qualcomm/README.md
@@ -1,3 +1,7 @@
+**Note that this script is archived and moved [here](https://github.com/mlcommons/cm4mlops/tree/main/script/app-mlperf-inference-qualcomm).**
+
+
+
 Automatically generated README for this automation recipe: **app-mlperf-inference-qualcomm**
 
 Category: **Modular MLPerf benchmarks**

diff --git a/cm-mlops/script/app-mlperf-inference/README.md b/cm-mlops/script/app-mlperf-inference/README.md
@@ -1,3 +1,7 @@
+**Note that this script is archived and moved [here](https://github.com/mlcommons/cm4mlops/tree/main/script/app-mlperf-inference).**
+
+
+
 Automatically generated README for this automation recipe: **app-mlperf-inference**
 
 Category: **Modular MLPerf inference benchmark pipeline**

diff --git a/cm-mlops/script/app-mlperf-training-nvidia/README.md b/cm-mlops/script/app-mlperf-training-nvidia/README.md
@@ -1,3 +1,7 @@
+**Note that this script is archived and moved [here](https://github.com/mlcommons/cm4mlops/tree/main/script/app-mlperf-training-nvidia).**
+
+
+
 Automatically generated README for this automation recipe: **app-mlperf-training-nvidia**
 
 Category: **Modular MLPerf training benchmark pipeline**

diff --git a/cm-mlops/script/app-mlperf-training-reference/README.md b/cm-mlops/script/app-mlperf-training-reference/README.md
@@ -1,3 +1,7 @@
+**Note that this script is archived and moved [here](https://github.com/mlcommons/cm4mlops/tree/main/script/app-mlperf-training-reference).**
+
+
+
 Automatically generated README for this automation recipe: **app-mlperf-training-reference**
 
 Category: **Modular MLPerf training benchmark pipeline**

diff --git a/cm-mlops/script/app-stable-diffusion-onnx-py/README.md b/cm-mlops/script/app-stable-diffusion-onnx-py/README.md
@@ -1,3 +1,7 @@
+**Note that this script is archived and moved [here](https://github.com/mlcommons/cm4mlops/tree/main/script/app-stable-diffusion-onnx-py).**
+
+
+
 Automatically generated README for this automation recipe: **app-stable-diffusion-onnx-py**
 
 Category: **Modular AI/ML application pipeline**

diff --git a/cm-mlops/script/benchmark-any-mlperf-inference-implementation/README.md b/cm-mlops/script/benchmark-any-mlperf-inference-implementation/README.md
@@ -1,3 +1,7 @@
+**Note that this script is archived and moved [here](https://github.com/mlcommons/cm4mlops/tree/main/script/benchmark-any-mlperf-inference-implementation).**
+
+
+
 Automatically generated README for this automation recipe: **benchmark-any-mlperf-inference-implementation**
 
 Category: **MLPerf benchmark support**

diff --git a/cm-mlops/script/benchmark-program-mlperf/README.md b/cm-mlops/script/benchmark-program-mlperf/README.md
@@ -1,3 +1,7 @@
+**Note that this script is archived and moved [here](https://github.com/mlcommons/cm4mlops/tree/main/script/benchmark-program-mlperf).**
+
+
+
 Automatically generated README for this automation recipe: **benchmark-program-mlperf**
 
 Category: **Modular MLPerf inference benchmark pipeline**

diff --git a/cm-mlops/script/benchmark-program/README.md b/cm-mlops/script/benchmark-program/README.md
@@ -1,3 +1,7 @@
+**Note that this script is archived and moved [here](https://github.com/mlcommons/cm4mlops/tree/main/script/benchmark-program).**
+
+
+
 Automatically generated README for this automation recipe: **benchmark-program**
 
 Category: **DevOps automation**

diff --git a/cm-mlops/script/build-docker-image/README.md b/cm-mlops/script/build-docker-image/README.md
@@ -1,3 +1,7 @@
+**Note that this script is archived and moved [here](https://github.com/mlcommons/cm4mlops/tree/main/script/build-docker-image).**
+
+
+
 Automatically generated README for this automation recipe: **build-docker-image**
 
 Category: **Docker automation**
@@ -153,6 +157,7 @@ ___
 #### New environment keys auto-detected from customize
 
 * `CM_DOCKER_BUILD_ARGS`
+* `CM_DOCKER_BUILD_CMD`
 * `CM_DOCKER_CACHE_ARG`
 * `CM_DOCKER_IMAGE_NAME`
 * `CM_DOCKER_IMAGE_REPO`

diff --git a/cm-mlops/script/build-dockerfile/README.md b/cm-mlops/script/build-dockerfile/README.md
@@ -1,3 +1,7 @@
+**Note that this script is archived and moved [here](https://github.com/mlcommons/cm4mlops/tree/main/script/build-dockerfile).**
+
+
+
 Automatically generated README for this automation recipe: **build-dockerfile**
 
 Category: **Docker automation**

diff --git a/cm-mlops/script/build-mlperf-inference-server-nvidia/README.md b/cm-mlops/script/build-mlperf-inference-server-nvidia/README.md
@@ -1,3 +1,7 @@
+**Note that this script is archived and moved [here](https://github.com/mlcommons/cm4mlops/tree/main/script/build-mlperf-inference-server-nvidia).**
+
+
+
 Automatically generated README for this automation recipe: **build-mlperf-inference-server-nvidia**
 
 Category: **MLPerf benchmark support**

diff --git a/cm-mlops/script/calibrate-model-for.qaic/README.md b/cm-mlops/script/calibrate-model-for.qaic/README.md
@@ -1,3 +1,7 @@
+**Note that this script is archived and moved [here](https://github.com/mlcommons/cm4mlops/tree/main/script/calibrate-model-for.qaic).**
+
+
+
 Automatically generated README for this automation recipe: **calibrate-model-for.qaic**
 
 Category: **AI/ML optimization**
@@ -251,8 +255,8 @@ ___
        - CM script: [get-preprocessed-dataset-squad](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/get-preprocessed-dataset-squad)
      * get,ml-model
        * CM names: `--adr.['model-src']...`
+       - CM script: [get-ml-model-abtf-ssd-pytorch](https://github.com/mlcommons/cm4abtf/tree/master/script/get-ml-model-abtf-ssd-pytorch)
        - CM script: [get-ml-model-3d-unet-kits19](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/get-ml-model-3d-unet-kits19)
-       - CM script: [get-ml-model-abtf-ssd-pytorch](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/get-ml-model-abtf-ssd-pytorch)
        - CM script: [get-ml-model-bert-base-squad](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/get-ml-model-bert-base-squad)
        - CM script: [get-ml-model-bert-large-squad](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/get-ml-model-bert-large-squad)
        - CM script: [get-ml-model-dlrm-terabyte](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/get-ml-model-dlrm-terabyte)

diff --git a/cm-mlops/script/compile-model-for.qaic/README.md b/cm-mlops/script/compile-model-for.qaic/README.md
@@ -1,3 +1,7 @@
+**Note that this script is archived and moved [here](https://github.com/mlcommons/cm4mlops/tree/main/script/compile-model-for.qaic).**
+
+
+
 Automatically generated README for this automation recipe: **compile-model-for.qaic**
 
 Category: **AI/ML optimization**
@@ -395,8 +399,8 @@ ___
        - CM script: [calibrate-model-for.qaic](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/calibrate-model-for.qaic)
      * get,ml-model
        * CM names: `--adr.['model-src']...`
+       - CM script: [get-ml-model-abtf-ssd-pytorch](https://github.com/mlcommons/cm4abtf/tree/master/script/get-ml-model-abtf-ssd-pytorch)
        - CM script: [get-ml-model-3d-unet-kits19](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/get-ml-model-3d-unet-kits19)
-       - CM script: [get-ml-model-abtf-ssd-pytorch](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/get-ml-model-abtf-ssd-pytorch)
        - CM script: [get-ml-model-bert-base-squad](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/get-ml-model-bert-base-squad)
        - CM script: [get-ml-model-bert-large-squad](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/get-ml-model-bert-large-squad)
        - CM script: [get-ml-model-dlrm-terabyte](https://github.com/mlcommons/ck/tree/master/cm-mlops/script/get-ml-model-dlrm-terabyte)

diff --git a/cm-mlops/script/compile-program/README.md b/cm-mlops/script/compile-program/README.md
@@ -1,3 +1,7 @@
+**Note that this script is archived and moved [here](https://github.com/mlcommons/cm4mlops/tree/main/script/compile-program).**
+
+
+
 Automatically generated README for this automation recipe: **compile-program**
 
 Category: **DevOps automation**

diff --git a/cm-mlops/script/convert-csv-to-md/README.md b/cm-mlops/script/convert-csv-to-md/README.md
@@ -1,3 +1,7 @@
+**Note that this script is archived and moved [here](https://github.com/mlcommons/cm4mlops/tree/main/script/convert-csv-to-md).**
+
+
+
 Automatically generated README for this automation recipe: **convert-csv-to-md**
 
 Category: **DevOps automation**

diff --git a/cm-mlops/script/convert-ml-model-huggingface-to-onnx/README.md b/cm-mlops/script/convert-ml-model-huggingface-to-onnx/README.md
@@ -1,3 +1,7 @@
+**Note that this script is archived and moved [here](https://github.com/mlcommons/cm4mlops/tree/main/script/convert-ml-model-huggingface-to-onnx).**
+
+
+
 Automatically generated README for this automation recipe: **convert-ml-model-huggingface-to-onnx**
 
 Category: **AI/ML models**

diff --git a/cm-mlops/script/copy-to-clipboard/README.md b/cm-mlops/script/copy-to-clipboard/README.md
@@ -1,3 +1,7 @@
+**Note that this script is archived and moved [here](https://github.com/mlcommons/cm4mlops/tree/main/script/copy-to-clipboard).**
+
+
+
 Automatically generated README for this automation recipe: **copy-to-clipboard**
 
 Category: **DevOps automation**

diff --git a/cm-mlops/script/create-conda-env/README.md b/cm-mlops/script/create-conda-env/README.md
@@ -1,3 +1,7 @@
+**Note that this script is archived and moved [here](https://github.com/mlcommons/cm4mlops/tree/main/script/create-conda-env).**
+
+
+
 Automatically generated README for this automation recipe: **create-conda-env**
 
 Category: **DevOps automation**

diff --git a/cm-mlops/script/create-fpgaconvnet-app-tinyml/README.md b/cm-mlops/script/create-fpgaconvnet-app-tinyml/README.md
@@ -1,3 +1,7 @@
+**Note that this script is archived and moved [here](https://github.com/mlcommons/cm4mlops/tree/main/script/create-fpgaconvnet-app-tinyml).**
+
+
+
 Automatically generated README for this automation recipe: **create-fpgaconvnet-app-tinyml**
 
 Category: **TinyML automation**