Template repository for benchmarking.

1. Go straight to scratch (first figure out where it is, on lumi you can type lumi-workspaces to see relevant scratch folders)... Because creating env in your home directory might result in a disk quota exceeded error. In some cases, disk storage is limited to 100K files and the environment cannot be created. Default uv cache to scratch too by adding those lines to your ~/.bashrc.

SCRATCH=<project_scratch_dir>
CACHE_DIR=$SCRATCH/.uv-cache
TMPDIR=$SCRATCH/.tmp

2. Clone the repository

git clone https://github.com/MeteoSwiss/mch-anemoi-benchmarks.git

3. Create a new branch (for example, lumi-proposal)

git switch --create <your-branch-name>

3. Install uv (see uv documentation for more details)

4. Sync uv environment

cd anemoi-hpc-benchmarks
uv venv -p 3.11
uv sync

1. Copy credentials from CSCS:

scp -r ~/.config/anemoi <username>@<hpc_address>://users/<username>/.config

2. Download data for emulator experiments from the anemoi catalog:

anemoi-datasets copy s3://ml-datasets/mch-co2-an-archive-0p02-2015-2020-6h-v3-pl13.zarr . --resume

and global 6-hourly data:

anemoi-datasets copy s3://ml-datasets/aifs-od-an-oper-0001-mars-n320-2016-2023-6h-v8.zarr . --resume

3. Download the data for the interpolator from the anemoi catalog:

anemoi-datasets copy s3://ml-datasets/mch-co2-an-archive-0p02-2015-2020-1h-v3-pl13.zarr . --resume

and global hourly data:

anemoi-datasets copy s3://ml-datasets/aifs-ea-an-oper-0001-mars-o96-1979-2022-1h-v4.zarr . --resume

1. Write benchmarking config your usecase, save them in config (stageC_benchmark, etc); limit training to 1 epoch and a portion of the dataset. Set sanity steps to 0 to avoid latency at the start of training.
2. Copy example_run_scaling.sh and adjust the config name. Run it.
3. Compute wall times for each node number and number of GPUs using elapsed_from_logs.py. The script will save the wall times in results text file.
4. Print benchmarks with scaling.py. Results should look like this: