From 5f3b4b9c7bf30ad9657c098a52aa5cd067371a4b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Michael=20J=C3=A4hn?= Date: Tue, 17 Dec 2024 12:32:10 +0100 Subject: [PATCH 01/18] Update Alps and vClusters --- docs/alps/index.md | 27 +++++---------------------- docs/alps/vclusters.md | 2 +- 2 files changed, 6 insertions(+), 23 deletions(-) diff --git a/docs/alps/index.md b/docs/alps/index.md index 9b95316f7..af1ddbe58 100644 --- a/docs/alps/index.md +++ b/docs/alps/index.md @@ -1,43 +1,26 @@ -!!! construction "Page under construction - last update: 2024-09-06" - - Information in this page is not yet complete nor final. It will be updated following the progress of - - - the Alps system deployment at CSCS - - C2SM's adaptation to this new system - # The Alps System Alps is a distributed HPC infrastructure managed by CSCS. Contrary to traditional HPC, it is composed of several logical units called vClusters (versatile clusters). From the users perspective, they play the role of a traditional HPC machine, each vCluster being tailored to the needs of a specific community. This setup also enables geographical distribution of vClusters which facilitates geo-redundancy. The main physical piece of Alps is hosted at CSCS in Lugano and a detailed description can be found at [their website :material-open-in-new:](https://www.cscs.ch/computers/alps){:target="_blank"}. ## vClusters -The following table shows the current plan for the final vClusters distribution on Alps at CSCS (not the current situation). +The following table shows current vClusters distribution on Alps at CSCS (only C2SM relevant vClusters are shown). | vCluster | Activity | Share | |----------|-------------------|------------------| -| Daint | User Lab | ~ 800 GH nodes | +| Santis | Weather & Climate | ~ 500 GH nodes | +| Daint | User Lab | ~ 600 GH nodes | | Clariden | Machine Learning | ~ 800 GH nodes | -| Santis | Weather & Climate | ~ 400 GH nodes | -| Tödi | Testing | few GH nodes | | Eiger | | multi-core nodes | *GH = Grace Hopper* -## Early Access - -For getting access to the vCluster dedicated to testing ([Tödi](vclusters.md/#todi){:target="_blank"}), CSCS offers [Preparatory Projects :material-open-in-new:](https://www.cscs.ch/user-lab/allocation-schemes/preparatory-projects){:target="_blank"}. - ## Support by CSCS -To contact CSCS staff directly, users can join their dedicated [Slack channel :material-open-in-new:](https://cscs-users.slack.com){:target="_blank"}. - -## File Systems +General information about access, file systems, vClusters, user environments and much more can be found at the [CSCS Knowledge Base](https://confluence.cscs.ch/display/KB). -!!! note "TODO" +To contact CSCS staff directly, users can join their dedicated [Slack workspace :material-open-in-new:](https://cscs-users.slack.com){:target="_blank"}, with dedicated channels for each vCluster. - - [ ] `/users`, `/store` and `/scratch` - - [ ] reserved space per vClsuter vs shared space - - [ ] ... ## Introductory Workshop Material diff --git a/docs/alps/vclusters.md b/docs/alps/vclusters.md index 366c9f329..af1e93547 100644 --- a/docs/alps/vclusters.md +++ b/docs/alps/vclusters.md @@ -1,4 +1,4 @@ -!!! construction "Page under construction - last update: 2024-09-06" +!!! construction "Page under construction - last update: 2024-12-17" Information in this page is not yet complete nor final. It will be updated following the progress of From 2acefa94f8ed5817146bb364e65603f24a96bf59 Mon Sep 17 00:00:00 2001 From: mjaehn Date: Tue, 17 Dec 2024 11:32:19 +0000 Subject: [PATCH 02/18] GitHub Action: Apply external link format --- docs/alps/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/alps/index.md b/docs/alps/index.md index af1ddbe58..2df21a571 100644 --- a/docs/alps/index.md +++ b/docs/alps/index.md @@ -17,7 +17,7 @@ The following table shows current vClusters distribution on Alps at CSCS (only C ## Support by CSCS -General information about access, file systems, vClusters, user environments and much more can be found at the [CSCS Knowledge Base](https://confluence.cscs.ch/display/KB). +General information about access, file systems, vClusters, user environments and much more can be found at the [CSCS Knowledge Base :material-open-in-new:](https://confluence.cscs.ch/display/KB){:target="_blank"}. To contact CSCS staff directly, users can join their dedicated [Slack workspace :material-open-in-new:](https://cscs-users.slack.com){:target="_blank"}, with dedicated channels for each vCluster. From b6f6132bf7d3918326095d243556a807a0f6b797 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Michael=20J=C3=A4hn?= Date: Tue, 17 Dec 2024 14:33:14 +0100 Subject: [PATCH 03/18] Update to current status, remove under construction --- docs/alps/uenvs.md | 9 +---- docs/alps/vclusters.md | 83 ++++++++++++++++++++++++------------------ 2 files changed, 49 insertions(+), 43 deletions(-) diff --git a/docs/alps/uenvs.md b/docs/alps/uenvs.md index b9106dd46..99f33320d 100644 --- a/docs/alps/uenvs.md +++ b/docs/alps/uenvs.md @@ -1,10 +1,3 @@ -!!! construction "Page under construction - last update: 2024-09-06" - - Information in this page is not yet complete nor final. It will be updated following the progress of - - - the Alps system deployment at CSCS - - C2SM's adaptation to this new system - # User Environments Software stacks at CSCS are now accessible through the so-called User Environments (uenv). They replace the previous monolithic software stack containing everything from which one could load any module, with all the infinite potential conflicts it involves. User environments contain the minimal software stack required for a certain activity, say, building and running ICON. They are generated by `spack`, packed into single `squashfs` file and then mounted by the user. In a way, they can be considered as poor man's containers. @@ -29,7 +22,7 @@ The user environments provided by CSCS are registered in a central database. In !!! warning - Old software stack images didn't have a mount point in the metadata which is now required for the new versions of the `uenv` tool and its `--uenv` slurm plugin counterpart. If you have images in your local repository that are older than roughly September 5th, please pull them again. It will only update the metadata + Old software stack images didn't have a mount point in the metadata which is now required for the new versions of the `uenv` tool and its `--uenv` slurm plugin counterpart. If you have images in your local repository that are older than roughly September 5th, please pull them again - it will only update the metadata. ## The `uenv` command line tool diff --git a/docs/alps/vclusters.md b/docs/alps/vclusters.md index af1e93547..39ddfa17b 100644 --- a/docs/alps/vclusters.md +++ b/docs/alps/vclusters.md @@ -1,10 +1,3 @@ -!!! construction "Page under construction - last update: 2024-12-17" - - Information in this page is not yet complete nor final. It will be updated following the progress of - - - the Alps system deployment at CSCS - - C2SM's adaptation to this new system - # Supported vClusters This page is hosting information about C2SM supported vClusters (not all CSCS vClusters). @@ -29,54 +22,74 @@ Host balfrin* daint* santis* todi* This would allow standard connections like `ssh santis` but also specifying the login node like `ssh santis-ln002` if needed. Replace `cscsusername` with your actual user name. -## Daint - -Daint (Alps) is the vCluster dedicated to the User Lab. It is currently accessible at `daint.alps.cscs.ch` (until the current Piz Daint gets decommissioned), so connect with `ssh daint.alps` with the `ssh` settings above. +## Santis -Even though Weather and Climate also has the dedicated vCluster Santis (see [below](#santis)), traditional projects might also land on Daint. +The vCluster `santis` is dedicated to **Climate and Weather**. It might, at the beginning, only host [EXCLAIM :material-open-in-new:](https://c2sm.ethz.ch/research/exclaim.html){:target="_blank"} and related projects. -### Uenvs +### Deployment Status -List of currently supported Uenvs on Daint: +By the end of 2024, the deployment is complete only to ~95 %. -| uenv | activity | Remark | -|--------------------------|--------------------------------|---------------------| -| icon-vx:rcy | build and run ICON | Not deployed (yet?) | -| netcdf-tools/2024:v1-rc1 | pre- and post-processing tools | | +### Differences to the Environment on `todi` -### Storage +- `$HOME` is now on a new NFS file system + - Your folder `/users/$USER` will be mostly empty + - The NFS system still requires some fine-tuning and file system performance might be low. + - We recommend to do (especially heavy tasks) on `$SCRATCH` +- The `$HOME` of `todi` is mounted in `/users.OLD/$USER` + - ⚠️ The mount is read-only! + - You are responsible to copy your data from `/users.OLD/$USER` :material-arrow-right-thin: `/users/$USER/...` + - The mount is temporary - will be removed towards end of January 2025. -!!! note "TODO" +!!! info - - [ ] Storage + Despite the need to work on the deployment in the upcoming days, users are invited to already access the system and start familiarising yourself with it and they might also start the data migration of your old home. -## Santis + The activities on CSCS side should not require any reboot, however, some services might need to be restarted, e.g., SLURM. This could lead to short interruptions or even failing jobs. CSCS will provide more information in the upcoming days and will try to minimise the risk of interferences by consolidating changes. -!!! warning "Santis has not been deployed yet." +### Uenvs -Santis is dedicated to Weather and Climate. It might, at the beginning, only host [EXCLAIM :material-open-in-new:](https://c2sm.ethz.ch/research/exclaim.html){:target="_blank"} and related projects. +To find and use already existing uenvs from `todi`, you need to modify the `CLUSTER_NAME` environment variable. -### Uenvs +```shell +export CLUSTER_NAME=todi +uenv image find +``` -| uenv | activity | -|--------------------------|--------------------------------| -| icon-vx:rcy | build and run ICON | -| netcdf-tools/tag:version | pre- and post-processing tools | +| uenv | activity | +|----------------------------|--------------------------------| +| `icon-wcp/v1:rc4` | build and run ICON | +| `netcdf-tools/2024:v1-rc1` | pre- and post-processing tools | ### Storage !!! note "TODO" - - [ ] Storage +The migration of the previous storage is not yet finished. Once there is an update from CSCS, we will inform you here. Also note that the environment variables `$STORE` and `$PROJECT` are not yet set. +## Daint -## Tödi +Daint (Alps) is the vCluster dedicated to the **User Lab**. It is currently accessible at `daint.alps.cscs.ch` (until the current Piz Daint gets decommissioned), so connect with `ssh daint.alps` with the `ssh` settings above. -Tödi is the testing vCluster and is currently deployed on the most of the Alps system. +Even though Climate and Weather also has the dedicated vCluster `santis` (see [below](#santis)), traditional projects might also land on Daint. ### Uenvs -| uenv | activity | -|--------------------------|--------------------------------| -| icon-wcp/v1:rc4 | build and run ICON | -| netcdf-tools/2024:v1-rc1 | pre- and post-processing tools | +Same as on `santis`, one can access the uenvs from `todi`: + +```shell +export CLUSTER_NAME=todi +uenv image find +``` + +| uenv | activity | +|----------------------------|--------------------------------| +| `icon-wcp/v1:rc4` | build and run ICON | +| `netcdf-tools/2024:v1-rc1` | pre- and post-processing tools | + +### Storage + +!!! note "TODO" + +The migration of the previous storage is not yet finished. Once there is an update from CSCS, we will inform you here. + From 9ba0e73f9abf70832de0262d153ae79bae89d5c3 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Michael=20J=C3=A4hn?= Date: Tue, 17 Dec 2024 14:43:01 +0100 Subject: [PATCH 04/18] Add news post --- docs/posts/2024-12-17_alps_update.md | 14 ++++++++++++++ 1 file changed, 14 insertions(+) create mode 100644 docs/posts/2024-12-17_alps_update.md diff --git a/docs/posts/2024-12-17_alps_update.md b/docs/posts/2024-12-17_alps_update.md new file mode 100644 index 000000000..8f28f387b --- /dev/null +++ b/docs/posts/2024-12-17_alps_update.md @@ -0,0 +1,14 @@ +--- +date: + created: 2024-12-17 +categories: + - Alps +--- + +# Update on the current status on Alps + +Last week, CSCS deployed the Climate and Weather vCluster `santis`. Since there is still some fine-tuning to be performed by CSCS, the [Santis section](../alps/vclusters.md#santis) provides an overview how to transition from `todi` to `santis`. + + + +Additionally, all information about the new vClusters as well as how to use User Environments (uenvs) have been updated. \ No newline at end of file From d1b888519df9cd7ea7c33ea352f683583250575e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Michael=20J=C3=A4hn?= Date: Tue, 17 Dec 2024 15:31:53 +0100 Subject: [PATCH 05/18] Update docs/alps/vclusters.md Co-authored-by: Annika Lauber --- docs/alps/vclusters.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/alps/vclusters.md b/docs/alps/vclusters.md index 39ddfa17b..4bad5e29c 100644 --- a/docs/alps/vclusters.md +++ b/docs/alps/vclusters.md @@ -20,7 +20,7 @@ Host balfrin* daint* santis* todi* ProxyJump ela ``` -This would allow standard connections like `ssh santis` but also specifying the login node like `ssh santis-ln002` if needed. Replace `cscsusername` with your actual user name. +This allows standard connections like `ssh santis`, but you can also specify a login node if needed, e.g., `ssh santis-ln002`. Replace `cscsusername` with your actual username. ## Santis From bb4eb0682595089bf2f5499fdb651d10e4ebe7ea Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Michael=20J=C3=A4hn?= Date: Tue, 17 Dec 2024 15:35:35 +0100 Subject: [PATCH 06/18] Update docs/alps/vclusters.md Co-authored-by: Annika Lauber --- docs/alps/vclusters.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/docs/alps/vclusters.md b/docs/alps/vclusters.md index 4bad5e29c..040daeb5b 100644 --- a/docs/alps/vclusters.md +++ b/docs/alps/vclusters.md @@ -33,13 +33,13 @@ By the end of 2024, the deployment is complete only to ~95 %. ### Differences to the Environment on `todi` - `$HOME` is now on a new NFS file system - - Your folder `/users/$USER` will be mostly empty - - The NFS system still requires some fine-tuning and file system performance might be low. - - We recommend to do (especially heavy tasks) on `$SCRATCH` -- The `$HOME` of `todi` is mounted in `/users.OLD/$USER` + - Your folder `/users/$USER` will initially be mostly empty + - The NFS system still requires fine-tuning, and file system performance may be low. + - We recommend running tasks, especially heavy ones, on $SCRATCH. +- `Todi`'s $HOME is mounted as `/users.OLD/$USER`. - ⚠️ The mount is read-only! - - You are responsible to copy your data from `/users.OLD/$USER` :material-arrow-right-thin: `/users/$USER/...` - - The mount is temporary - will be removed towards end of January 2025. + - You are responsible for copying your data from `/users.OLD/$USER` to `/users/$USER/...`. + - The mount is temporary and will be removed by the end of January 2025. !!! info From a63b11b10ef4400cc2e7e563f591e775ddeca5b4 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Michael=20J=C3=A4hn?= Date: Tue, 17 Dec 2024 15:36:02 +0100 Subject: [PATCH 07/18] Update docs/alps/vclusters.md Co-authored-by: Annika Lauber --- docs/alps/vclusters.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/alps/vclusters.md b/docs/alps/vclusters.md index 040daeb5b..38cc32b6f 100644 --- a/docs/alps/vclusters.md +++ b/docs/alps/vclusters.md @@ -43,7 +43,7 @@ By the end of 2024, the deployment is complete only to ~95 %. !!! info - Despite the need to work on the deployment in the upcoming days, users are invited to already access the system and start familiarising yourself with it and they might also start the data migration of your old home. + Despite the need to work on the deployment in the upcoming days, users are invited to already access the system and start familiarising themselves with it and they might also start the data migration of their old home. The activities on CSCS side should not require any reboot, however, some services might need to be restarted, e.g., SLURM. This could lead to short interruptions or even failing jobs. CSCS will provide more information in the upcoming days and will try to minimise the risk of interferences by consolidating changes. From 6580ad601535a9b89b72db478bc76133c219731c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Michael=20J=C3=A4hn?= Date: Tue, 17 Dec 2024 15:37:12 +0100 Subject: [PATCH 08/18] Update docs/alps/vclusters.md Co-authored-by: Annika Lauber --- docs/alps/vclusters.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/alps/vclusters.md b/docs/alps/vclusters.md index 38cc32b6f..1a9fef435 100644 --- a/docs/alps/vclusters.md +++ b/docs/alps/vclusters.md @@ -75,7 +75,7 @@ Even though Climate and Weather also has the dedicated vCluster `santis` (see [b ### Uenvs -Same as on `santis`, one can access the uenvs from `todi`: +As on `santis`, you can access the uenvs from `todi`: ```shell export CLUSTER_NAME=todi From ca5836dc489d7a680e8e39197993422e07801749 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Michael=20J=C3=A4hn?= Date: Tue, 17 Dec 2024 15:41:23 +0100 Subject: [PATCH 09/18] Update docs/posts/2024-12-17_alps_update.md Co-authored-by: Annika Lauber --- docs/posts/2024-12-17_alps_update.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/posts/2024-12-17_alps_update.md b/docs/posts/2024-12-17_alps_update.md index 8f28f387b..c97c1820f 100644 --- a/docs/posts/2024-12-17_alps_update.md +++ b/docs/posts/2024-12-17_alps_update.md @@ -7,7 +7,7 @@ categories: # Update on the current status on Alps -Last week, CSCS deployed the Climate and Weather vCluster `santis`. Since there is still some fine-tuning to be performed by CSCS, the [Santis section](../alps/vclusters.md#santis) provides an overview how to transition from `todi` to `santis`. +Last week, CSCS deployed the Climate and Weather vCluster `santis`. As some fine-tuning is still ongoing, the [Santis section](../alps/vclusters.md#santis) provides an overview of how to transition from `todi` to `santis`. From a7472fd08c4a400b73c5900dae97f0540c749ab5 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Michael=20J=C3=A4hn?= Date: Tue, 17 Dec 2024 15:45:28 +0100 Subject: [PATCH 10/18] Update docs/alps/vclusters.md Co-authored-by: Annika Lauber --- docs/alps/vclusters.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/alps/vclusters.md b/docs/alps/vclusters.md index 1a9fef435..03d8cf8e8 100644 --- a/docs/alps/vclusters.md +++ b/docs/alps/vclusters.md @@ -24,7 +24,7 @@ This allows standard connections like `ssh santis`, but you can also specify a l ## Santis -The vCluster `santis` is dedicated to **Climate and Weather**. It might, at the beginning, only host [EXCLAIM :material-open-in-new:](https://c2sm.ethz.ch/research/exclaim.html){:target="_blank"} and related projects. +The vCluster `Santis` is dedicated to **Climate and Weather** and may initially host only [EXCLAIM :material-open-in-new:](https://c2sm.ethz.ch/research/exclaim.html){:target="_blank"} and related projects. ### Deployment Status From e8c87c29643956247318b96db25d644ccbaa186f Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Michael=20J=C3=A4hn?= Date: Tue, 17 Dec 2024 15:49:10 +0100 Subject: [PATCH 11/18] Update vclusters --- docs/alps/vclusters.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/alps/vclusters.md b/docs/alps/vclusters.md index 03d8cf8e8..be79312cc 100644 --- a/docs/alps/vclusters.md +++ b/docs/alps/vclusters.md @@ -28,7 +28,7 @@ The vCluster `Santis` is dedicated to **Climate and Weather** and may initially ### Deployment Status -By the end of 2024, the deployment is complete only to ~95 %. +Currently, the deployment is approximately 95% complete. ### Differences to the Environment on `todi` From b75a152859a5243cd03971c59a22d1b68a697724 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Michael=20J=C3=A4hn?= Date: Tue, 17 Dec 2024 15:51:38 +0100 Subject: [PATCH 12/18] Unify vCluster names --- docs/alps/vclusters.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/alps/vclusters.md b/docs/alps/vclusters.md index be79312cc..c7c64570b 100644 --- a/docs/alps/vclusters.md +++ b/docs/alps/vclusters.md @@ -24,7 +24,7 @@ This allows standard connections like `ssh santis`, but you can also specify a l ## Santis -The vCluster `Santis` is dedicated to **Climate and Weather** and may initially host only [EXCLAIM :material-open-in-new:](https://c2sm.ethz.ch/research/exclaim.html){:target="_blank"} and related projects. +The vCluster `santis` is dedicated to **Climate and Weather** and may initially host only [EXCLAIM :material-open-in-new:](https://c2sm.ethz.ch/research/exclaim.html){:target="_blank"} and related projects. ### Deployment Status @@ -36,7 +36,7 @@ Currently, the deployment is approximately 95% complete. - Your folder `/users/$USER` will initially be mostly empty - The NFS system still requires fine-tuning, and file system performance may be low. - We recommend running tasks, especially heavy ones, on $SCRATCH. -- `Todi`'s $HOME is mounted as `/users.OLD/$USER`. +- `todi`'s $HOME is mounted as `/users.OLD/$USER`. - ⚠️ The mount is read-only! - You are responsible for copying your data from `/users.OLD/$USER` to `/users/$USER/...`. - The mount is temporary and will be removed by the end of January 2025. From 36188fc4730684bb1f3bd2cccdc1e111e1205d05 Mon Sep 17 00:00:00 2001 From: Annika Lauber Date: Wed, 18 Dec 2024 11:24:45 +0100 Subject: [PATCH 13/18] =?UTF-8?q?Remove=20T=C3=B6di=20from=20install=20ins?= =?UTF-8?q?tructions?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- docs/models/icon/usage.md | 25 ++++++++++++------------- 1 file changed, 12 insertions(+), 13 deletions(-) diff --git a/docs/models/icon/usage.md b/docs/models/icon/usage.md index 4963f088c..b60f824d6 100644 --- a/docs/models/icon/usage.md +++ b/docs/models/icon/usage.md @@ -69,13 +69,19 @@ srun -N 1 -c 12 --mem-per-cpu=20G spack install -v -j 12 ``` -### Todi +### Säntis -!!! construction "Under construction - last update: 2024-09-20" +!!! construction "Under construction - last update: 2024-12-18" Information on this section is not yet complete nor final. It will be updated following the progress of the Alps system deployment at CSCS and C2SM's adaptation to this new system. Please use the [C2SM support forum :material-open-in-new:](https://github.com/C2SM/Tasks-Support/discussions){:target="_blank"} in case of questions regarding building ICON on Alps. -On Todi, Spack is also used to build ICON. However, there is no suitable `spack.yaml` file present for the Spack environment. Therefore, create a `spack.yaml` file and use the software stack upstream provided by the user environment. +Currently, the same ICON user environment used on `todi` is being used. Since the environment is still linked to `todi`, you need to export the `CLUSTER_NAME` to `todi` for now: + +```bash +export CLUSTER_NAME=todi +``` + +Next, follow the instructions to build ICON using Spack below. **1. Create a `spack.yaml` file** @@ -83,7 +89,7 @@ Create the following files from the ICON build folder (different to the ICON roo For CPU compilation: -=== "config/cscs/spack/v0.21.1.3/todi_cpu_nvhpc/spack.yaml" +=== "config/cscs/spack/v0.21.1.3/santis_cpu_nvhpc/spack.yaml" ```yaml spack: @@ -101,7 +107,7 @@ For CPU compilation: For GPU compilation: -=== "config/cscs/spack/v0.21.1.3/todi_gpu_nvhpc/spack.yaml" +=== "config/cscs/spack/v0.21.1.3/santis_gpu_nvhpc/spack.yaml" ```yaml spack: @@ -131,18 +137,11 @@ git clone --depth 1 --recurse-submodules --shallow-submodules -b ${SPACK_TAG} ht # Build ICON cd /path/to/icon-build-folder -spack env activate -d config/cscs/spack/${SPACK_TAG}/todi_gpu_nvhpc +spack env activate -d config/cscs/spack/${SPACK_TAG}/santis_gpu_nvhpc spack install ``` -### Santis -Please follow the instructions for Todi, but run the following before loading the ICON user-environment: - -```bash -export CLUSTER_NAME=todi -``` - ## Run test case In the *run* folder, you find many prepared test cases, which you can convert into run scripts. To generate the runscript of one of the experiment files, e.g. *mch_ch_lowres*, you can use the `make_runscripts` function. From a6b029da26347a271dd24797cb8a18f4fe1e9a0a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Michael=20J=C3=A4hn?= Date: Wed, 18 Dec 2024 11:39:10 +0100 Subject: [PATCH 14/18] Use todi for spack sysconfig --- docs/models/icon/usage.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/models/icon/usage.md b/docs/models/icon/usage.md index b60f824d6..4a2613324 100644 --- a/docs/models/icon/usage.md +++ b/docs/models/icon/usage.md @@ -89,7 +89,7 @@ Create the following files from the ICON build folder (different to the ICON roo For CPU compilation: -=== "config/cscs/spack/v0.21.1.3/santis_cpu_nvhpc/spack.yaml" +=== "config/cscs/spack/v0.21.1.3/todi_cpu_nvhpc/spack.yaml" ```yaml spack: @@ -107,7 +107,7 @@ For CPU compilation: For GPU compilation: -=== "config/cscs/spack/v0.21.1.3/santis_gpu_nvhpc/spack.yaml" +=== "config/cscs/spack/v0.21.1.3/todi_gpu_nvhpc/spack.yaml" ```yaml spack: @@ -137,7 +137,7 @@ git clone --depth 1 --recurse-submodules --shallow-submodules -b ${SPACK_TAG} ht # Build ICON cd /path/to/icon-build-folder -spack env activate -d config/cscs/spack/${SPACK_TAG}/santis_gpu_nvhpc +spack env activate -d config/cscs/spack/${SPACK_TAG}/todi_gpu_nvhpc spack install ``` From 26f1377a3764f87d5f3e5deca6742b51c3fd3d8e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Michael=20J=C3=A4hn?= Date: Wed, 18 Dec 2024 11:42:30 +0100 Subject: [PATCH 15/18] Rename todi to alps --- docs/models/icon/usage.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/models/icon/usage.md b/docs/models/icon/usage.md index 4a2613324..adef14088 100644 --- a/docs/models/icon/usage.md +++ b/docs/models/icon/usage.md @@ -89,7 +89,7 @@ Create the following files from the ICON build folder (different to the ICON roo For CPU compilation: -=== "config/cscs/spack/v0.21.1.3/todi_cpu_nvhpc/spack.yaml" +=== "config/cscs/spack/v0.21.1.3/alps_cpu_nvhpc/spack.yaml" ```yaml spack: @@ -107,7 +107,7 @@ For CPU compilation: For GPU compilation: -=== "config/cscs/spack/v0.21.1.3/todi_gpu_nvhpc/spack.yaml" +=== "config/cscs/spack/v0.21.1.3/alps_gpu_nvhpc/spack.yaml" ```yaml spack: @@ -137,7 +137,7 @@ git clone --depth 1 --recurse-submodules --shallow-submodules -b ${SPACK_TAG} ht # Build ICON cd /path/to/icon-build-folder -spack env activate -d config/cscs/spack/${SPACK_TAG}/todi_gpu_nvhpc +spack env activate -d config/cscs/spack/${SPACK_TAG}/alps_gpu_nvhpc spack install ``` From 593348e85a6c72ba64c99836bff2a82cb2f600b0 Mon Sep 17 00:00:00 2001 From: Annika Lauber Date: Wed, 18 Dec 2024 11:49:02 +0100 Subject: [PATCH 16/18] Add instructions to create missing folders --- docs/models/icon/usage.md | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/docs/models/icon/usage.md b/docs/models/icon/usage.md index adef14088..435e32c0a 100644 --- a/docs/models/icon/usage.md +++ b/docs/models/icon/usage.md @@ -85,7 +85,12 @@ Next, follow the instructions to build ICON using Spack below. **1. Create a `spack.yaml` file** -Create the following files from the ICON build folder (different to the ICON root folder in case of a out-of-source build). +Create the following files from the ICON build folder (different to the ICON root folder in case of a out-of-source build). For that, you will have to create the missing folders first: +```bash +mkdir config/cscs/spack/v0.21.1.3 +mkdir config/cscs/spack/v0.21.1.3/alps_cpu_nvhpc +mkdir config/cscs/spack/v0.21.1.3/alps_gpu_nvhpc +``` For CPU compilation: From 882e487d930d21092e2c0a61cf23962a0245a5b9 Mon Sep 17 00:00:00 2001 From: Annika Lauber Date: Wed, 18 Dec 2024 11:51:07 +0100 Subject: [PATCH 17/18] Swap santis with daint --- docs/models/icon/usage.md | 110 +++++++++++++++++++------------------- 1 file changed, 54 insertions(+), 56 deletions(-) diff --git a/docs/models/icon/usage.md b/docs/models/icon/usage.md index 435e32c0a..83ced7491 100644 --- a/docs/models/icon/usage.md +++ b/docs/models/icon/usage.md @@ -13,62 +13,6 @@ Once you have access, clone the repository from GitHub using the SSH protocol: ## Configure and compile -### Piz Daint -Spack is used to build ICON. Please follow the steps below to set up Spack and build ICON. - -**1. Set up a Spack instance** - -To [set up a Spack instance :material-open-in-new:](https://c2sm.github.io/spack-c2sm/latest/QuickStart.html#at-cscs-daint-tsa-balfrin){:target="_blank"}, ensure that you clone the repository using the Spack tag provided in the ICON repository at [config/cscs/SPACK_TAG_C2SM :material-open-in-new:](https://github.com/C2SM/icon/blob/main/config/cscs/SPACK_TAG_C2SM){:target="_blank"} and load it into your command line. - -**2. Build ICON** - -Refer to the official spack-c2sm documentation for [installing ICON using Spack :material-open-in-new:](https://c2sm.github.io/spack-c2sm/latest/QuickStart.html#icon){:target="_blank"}. - -After the first compilation, you need to create a `setting` file (the following example is for Piz Daint, please adapt the lines according to the machine you are using): - -=== "daint_gpu_nvhpc" - ```shell - # Get SPACK_TAG used on machine - SPACK_TAG=$(cat "config/cscs/SPACK_TAG_C2SM") - # Set the name of the environment, which should be equal to the builder - ENV_NAME=daint_gpu_nvhpc - # Load probtest environment (only needed if you want to run check files) - source /project/g110/icon/probtest/conda/miniconda/bin/activate probtest - # Ensure CDO is loaded on your machine - module load daint-gpu CDO - # Remove and create setting file with the following two commands - rm -f setting - ./config/cscs/create_sh_env $SPACK_TAG $ENV_NAME - ``` - -### Euler -Spack is used to build ICON. Please follow the steps below to set up Spack and build ICON. - -**1. Set up a Spack instance** - -To [set up a Spack instance :material-open-in-new:](https://c2sm.github.io/spack-c2sm/latest/QuickStart.html#at-cscs-daint-tsa-balfrin){:target="_blank"}, ensure that you clone the repository using the Spack tag provided in the ICON repository at [config/ethz/SPACK_TAG_EULER :material-open-in-new:](https://github.com/C2SM/icon/blob/main/config/ethz/SPACK_TAG_EULER){:target="_blank"} and load it into your command line. - - -**2. Build ICON** - -Activate the Spack environment for Euler: -```bash -SPACK_TAG=$(cat "config/ethz/SPACK_TAG_EULER") -spack env activate -d config/ethz/spack/$SPACK_TAG/euler_cpu_gcc -``` - -Euler Support recommends to compile code on compute-nodes. Unfortunately [internet-access on Euler compute-nodes is restricted :material-open-in-new:](https://scicomp.ethz.ch/wiki/Accessing_the_clusters#Internet_Security){:target="_blank"}. -Therefore a two-step install needs to be performed: - -```bash -# fetch and install cosmo-eccodes-definitions on login-node -spack install cosmo-eccodes-definitions - -# compile ICON on compute-nodes -srun -N 1 -c 12 --mem-per-cpu=20G spack install -v -j 12 -``` - - ### Säntis !!! construction "Under construction - last update: 2024-12-18" @@ -146,6 +90,60 @@ spack env activate -d config/cscs/spack/${SPACK_TAG}/alps_gpu_nvhpc spack install ``` +### Euler +Spack is used to build ICON. Please follow the steps below to set up Spack and build ICON. + +**1. Set up a Spack instance** + +To [set up a Spack instance :material-open-in-new:](https://c2sm.github.io/spack-c2sm/latest/QuickStart.html#at-cscs-daint-tsa-balfrin){:target="_blank"}, ensure that you clone the repository using the Spack tag provided in the ICON repository at [config/ethz/SPACK_TAG_EULER :material-open-in-new:](https://github.com/C2SM/icon/blob/main/config/ethz/SPACK_TAG_EULER){:target="_blank"} and load it into your command line. + + +**2. Build ICON** + +Activate the Spack environment for Euler: +```bash +SPACK_TAG=$(cat "config/ethz/SPACK_TAG_EULER") +spack env activate -d config/ethz/spack/$SPACK_TAG/euler_cpu_gcc +``` + +Euler Support recommends to compile code on compute-nodes. Unfortunately [internet-access on Euler compute-nodes is restricted :material-open-in-new:](https://scicomp.ethz.ch/wiki/Accessing_the_clusters#Internet_Security){:target="_blank"}. +Therefore a two-step install needs to be performed: + +```bash +# fetch and install cosmo-eccodes-definitions on login-node +spack install cosmo-eccodes-definitions + +# compile ICON on compute-nodes +srun -N 1 -c 12 --mem-per-cpu=20G spack install -v -j 12 +``` + +### Piz Daint +Spack is used to build ICON. Please follow the steps below to set up Spack and build ICON. + +**1. Set up a Spack instance** + +To [set up a Spack instance :material-open-in-new:](https://c2sm.github.io/spack-c2sm/latest/QuickStart.html#at-cscs-daint-tsa-balfrin){:target="_blank"}, ensure that you clone the repository using the Spack tag provided in the ICON repository at [config/cscs/SPACK_TAG_C2SM :material-open-in-new:](https://github.com/C2SM/icon/blob/main/config/cscs/SPACK_TAG_C2SM){:target="_blank"} and load it into your command line. + +**2. Build ICON** + +Refer to the official spack-c2sm documentation for [installing ICON using Spack :material-open-in-new:](https://c2sm.github.io/spack-c2sm/latest/QuickStart.html#icon){:target="_blank"}. + +After the first compilation, you need to create a `setting` file (the following example is for Piz Daint, please adapt the lines according to the machine you are using): + +=== "daint_gpu_nvhpc" + ```shell + # Get SPACK_TAG used on machine + SPACK_TAG=$(cat "config/cscs/SPACK_TAG_C2SM") + # Set the name of the environment, which should be equal to the builder + ENV_NAME=daint_gpu_nvhpc + # Load probtest environment (only needed if you want to run check files) + source /project/g110/icon/probtest/conda/miniconda/bin/activate probtest + # Ensure CDO is loaded on your machine + module load daint-gpu CDO + # Remove and create setting file with the following two commands + rm -f setting + ./config/cscs/create_sh_env $SPACK_TAG $ENV_NAME + ``` ## Run test case In the *run* folder, you find many prepared test cases, which you can convert into run scripts. To generate the runscript of one of the experiment files, e.g. *mch_ch_lowres*, you can use the `make_runscripts` function. From 1d3b892718762b2465318386307e95e4ee1aa9ef Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Michael=20J=C3=A4hn?= Date: Wed, 18 Dec 2024 12:01:12 +0100 Subject: [PATCH 18/18] mkdir -p --- docs/models/icon/usage.md | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/docs/models/icon/usage.md b/docs/models/icon/usage.md index 83ced7491..09699c45a 100644 --- a/docs/models/icon/usage.md +++ b/docs/models/icon/usage.md @@ -31,9 +31,8 @@ Next, follow the instructions to build ICON using Spack below. Create the following files from the ICON build folder (different to the ICON root folder in case of a out-of-source build). For that, you will have to create the missing folders first: ```bash -mkdir config/cscs/spack/v0.21.1.3 -mkdir config/cscs/spack/v0.21.1.3/alps_cpu_nvhpc -mkdir config/cscs/spack/v0.21.1.3/alps_gpu_nvhpc +mkdir -p config/cscs/spack/v0.21.1.3/alps_cpu_nvhpc +mkdir -p config/cscs/spack/v0.21.1.3/alps_gpu_nvhpc ``` For CPU compilation: