From 76a0504b2d641fe8ab7e191686adc6a2b23bde3b Mon Sep 17 00:00:00 2001 From: Caspar van Leeuwen Date: Wed, 6 Aug 2025 17:30:15 +0200 Subject: [PATCH 1/5] Update bot build instructions for https://github.com/EESSI/eessi-bot-software-layer/pull/312 --- docs/bot.md | 54 ++++++++++++++++++++++++++++++++++++++--------------- 1 file changed, 39 insertions(+), 15 deletions(-) diff --git a/docs/bot.md b/docs/bot.md index 25eadac16c..9b3ee620b3 100644 --- a/docs/bot.md +++ b/docs/bot.md @@ -66,13 +66,15 @@ should be issued by posting a comment in the pull request (see also [here](addin The most basic build instruction that can be sent to the bot is: ``` -bot: build +bot: build for:arch= ``` -!!! warning - Only use `bot: build` if you are confident that it is OK to do so. +Where the `for_arch` could be e.g. `x64_64/amd/zen4`. This will trigger the bot to allocate a node of that type _and_ +build in the `/cvmfs/software.eessi.io/versions//software/linux/x86_64/amd/zen4` prefix. - Most likely, you want to supply one or more filters to avoid that the bot builds for all its configurations. +!!! warning + Most likely, you want to supply one or more filters to avoid that all bots are triggered to build for all + configurations that match the above command. ### Filters @@ -81,13 +83,14 @@ should be executed, based on: - `instance`: the `name` of the bot instance, for example `instance:aws` for the bot instance running in AWS; - `repository`: the target repository, for example `eessi-2023.06-software` which corresponds to the 2023.06 version of the EESSI software layer; -- `architecture`: the name of the [CPU microarchitecture](software_layer/cpu_targets.md), for example `x86_64/amd/zen2`; +- `on:architecture=,accelerator=`: the name of the [CPU microarchitecture](software_layer/cpu_targets.md) and GPU accelerator you want to build _on_, for example `for:architecture=x86_64/amd/zen4,accelerator=nvidia/cc90`; +- `for:architecture=,accelerator=`: the name of the [CPU microarchitecture](software_layer/cpu_targets.md) and GPU accelerator you want to build _for_, for example `on:architecture=x86_64/amd/zen4,accelerator=nvidia/cc90`; !!! note - Use `:` as separator to specify a value for a particular filter, do not add spaces after the `:`. + Use `:` as separator to specify a value for a particular argument, do not add spaces after the `:`. The bot recognizes shorthands for the supported filters, so you can use `inst:...` instead of `instance:...`, - `repo:...` instead of `repository:...`, and `arch:...` instead of `architecture:...`. + `repo:...` instead of `repository:...`, and `arch=...` instead of `architecture=...`, and `accel=` instead of `accelerator`. #### Combining filters @@ -97,7 +100,7 @@ Separate filters with a space, order of filters does not matter. For example: ``` -bot: build repo:eessi.io-2023.06-software arch:x86_64/amd/zen2 +bot: build repo:eessi.io-2023.06-software for:arch=x86_64/amd/zen2 ``` #### Multiple build instructions @@ -108,17 +111,38 @@ repositories, and CPU targets. Specify one build instruction per line. For example: ``` -bot: build repo:eessi.io-2023.06-software arch:x86_64/amd/zen3 inst:aws -bot: build repo:eessi.io-2023.06-software arch:x86_64/amd/zen4 inst:azure +bot: build repo:eessi.io-2023.06-software for:arch=x86_64/amd/zen3 inst:aws +bot: build repo:eessi.io-2023.06-software for:arch=x86_64/amd/zen4 inst:azure ``` -!!! note - The bot applies the filters with partial matching, which you can use to combine multiple build - instructions into a single one. +#### Native builds + +If you want to allocate the same node type that you want to build for, you can ommit the `on:` argument. +For example: `bot:build for:arch=x86_64/amd/zen4` is fully equivalent to `bot:build on:arch=x86_64/amd/zen4 for:arch=x86_64/amd/zen4`. - For example, if you only want to build for all `aarch64` CPU targets, you can use `arch:aarch64` as filter. +#### Cross-compiling - The same applies to the `instance` and `repository` filters. +The reason for the separate `on:` and `for:` arguments to exist is to allow cross-compilation, and to be specific about +which architecture to allocate when doing so. The typical use case is to build GPU software on a CPU-only node. + +For example: `bot:build on:arch=x86_64/amd/zen2 for:arch=x86_64/amd/zen2,accel=nvidia/cc80` will instruct the bot to +build on a `zen2` CPU-only node, for a combination of a `zen2` CPU with a GPU with CUDA Compute Capability 8.0. + +!!! warning + Cross-compilation for different CPU targets can _not_ be done with the current setup. This is not a limitation of + the bot, but of the build scripts in the `software-layer-scripts` repository. The only thing the bot does is + prepare a job directory in which the configuration passed through `for:` is stored in a `job.cfg` file. What is + _done_ with that information is up to the build scripts from `software-layer-scripts`. While these currently + set the CUDA Compute Capability configuration item for EasyBuild based on the accelerator target defined in the + `job.cfg`, the architecture target is _only_ used to determine the installation path. It is _not_ used to set + EasyBuild's `optarch` configuration, which will still default to native optimization (i.e. for the host). + +#### Partial filter matching +The bot applies the filters with partial matching, but _not_ for the `for:` argument. I.e. you can do +`bot:build on:arch=zen4 for:arch=x84_64/amd/zen4,accel=nvidia/cc80`, but not +`bot:build on:arch=zen4 for:arch=zen4,accel=nvidia/cc80` +The reason is that for the `on:` argument, the bot can compare against the configured node types to find a match +(while for the `for:` argument, there is no such reference to match against). ### Behind-the-scenes From 8934f590b1460b846ac84457cba222e1c513de03 Mon Sep 17 00:00:00 2001 From: Caspar van Leeuwen Date: Wed, 6 Aug 2025 17:33:44 +0200 Subject: [PATCH 2/5] Add special behavior for accelerators --- docs/bot.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/docs/bot.md b/docs/bot.md index 9b3ee620b3..9d98311622 100644 --- a/docs/bot.md +++ b/docs/bot.md @@ -144,6 +144,11 @@ The bot applies the filters with partial matching, but _not_ for the `for:` argu The reason is that for the `on:` argument, the bot can compare against the configured node types to find a match (while for the `for:` argument, there is no such reference to match against). +#### Accelerator filter matching +If the bot config declares that a node type has a certain accelerator, that node type will _only_ be allocated +if a corresponding `accel=` argument is passed. This is to avoid that `on:arch=x86_64/amd/zen4` would +cause builds to be triggered on a `zen4`+GPU node, while the same system also has a CPU-only `zen4` node type. + ### Behind-the-scenes #### Processing build instructions From 89338ee2782f7e1ae10760e103e156b4427cd832 Mon Sep 17 00:00:00 2001 From: Caspar van Leeuwen <33718780+casparvl@users.noreply.github.com> Date: Thu, 7 Aug 2025 09:56:53 +0200 Subject: [PATCH 3/5] Fix typo --- docs/bot.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/bot.md b/docs/bot.md index 9d98311622..66ab567e08 100644 --- a/docs/bot.md +++ b/docs/bot.md @@ -117,7 +117,7 @@ bot: build repo:eessi.io-2023.06-software for:arch=x86_64/amd/zen4 inst:azure #### Native builds -If you want to allocate the same node type that you want to build for, you can ommit the `on:` argument. +If you want to allocate the same node type that you want to build for, you can omit the `on:` argument. For example: `bot:build for:arch=x86_64/amd/zen4` is fully equivalent to `bot:build on:arch=x86_64/amd/zen4 for:arch=x86_64/amd/zen4`. #### Cross-compiling From 50589498daacf72ebe2972d566de1c43b5062d2c Mon Sep 17 00:00:00 2001 From: Caspar van Leeuwen Date: Thu, 7 Aug 2025 11:39:38 +0200 Subject: [PATCH 4/5] Added note on for and on arguments being supported from bot v 0.9.0 onwards --- docs/bot.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/docs/bot.md b/docs/bot.md index 66ab567e08..092190991f 100644 --- a/docs/bot.md +++ b/docs/bot.md @@ -72,6 +72,9 @@ bot: build for:arch= Where the `for_arch` could be e.g. `x64_64/amd/zen4`. This will trigger the bot to allocate a node of that type _and_ build in the `/cvmfs/software.eessi.io/versions//software/linux/x86_64/amd/zen4` prefix. +!!! note + The `for:` (and `on:`, see below) argument to the bot where introduced in bot version 0.9.0. They replace the `architecture=... accelerator=...` syntax used in bot versions <= v0.8.0. + !!! warning Most likely, you want to supply one or more filters to avoid that all bots are triggered to build for all configurations that match the above command. From bfabf125de2a559337c996494070b9547f07e4a3 Mon Sep 17 00:00:00 2001 From: Caspar van Leeuwen <33718780+casparvl@users.noreply.github.com> Date: Mon, 11 Aug 2025 11:31:48 +0200 Subject: [PATCH 5/5] Apply suggestions from code review by bedroge MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Bob Dröge --- docs/bot.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/bot.md b/docs/bot.md index 092190991f..9bfd9786bf 100644 --- a/docs/bot.md +++ b/docs/bot.md @@ -73,7 +73,7 @@ Where the `for_arch` could be e.g. `x64_64/amd/zen4`. This will trigger the bot build in the `/cvmfs/software.eessi.io/versions//software/linux/x86_64/amd/zen4` prefix. !!! note - The `for:` (and `on:`, see below) argument to the bot where introduced in bot version 0.9.0. They replace the `architecture=... accelerator=...` syntax used in bot versions <= v0.8.0. + The `for:` (and `on:`, see below) argument to the bot were introduced in bot version 0.9.0. They replace the `architecture=... accelerator=...` syntax used in bot versions <= v0.8.0. !!! warning Most likely, you want to supply one or more filters to avoid that all bots are triggered to build for all @@ -86,8 +86,8 @@ should be executed, based on: - `instance`: the `name` of the bot instance, for example `instance:aws` for the bot instance running in AWS; - `repository`: the target repository, for example `eessi-2023.06-software` which corresponds to the 2023.06 version of the EESSI software layer; -- `on:architecture=,accelerator=`: the name of the [CPU microarchitecture](software_layer/cpu_targets.md) and GPU accelerator you want to build _on_, for example `for:architecture=x86_64/amd/zen4,accelerator=nvidia/cc90`; -- `for:architecture=,accelerator=`: the name of the [CPU microarchitecture](software_layer/cpu_targets.md) and GPU accelerator you want to build _for_, for example `on:architecture=x86_64/amd/zen4,accelerator=nvidia/cc90`; +- `on:architecture=,accelerator=`: the name of the [CPU microarchitecture](software_layer/cpu_targets.md) and GPU accelerator you want to build _on_, for example `on:architecture=x86_64/amd/zen4,accelerator=nvidia/cc90`; +- `for:architecture=,accelerator=`: the name of the [CPU microarchitecture](software_layer/cpu_targets.md) and GPU accelerator you want to build _for_, for example `for:architecture=x86_64/amd/zen4,accelerator=nvidia/cc90`; !!! note Use `:` as separator to specify a value for a particular argument, do not add spaces after the `:`.