From db8562479bf7b7de654913041c5a4186294de098 Mon Sep 17 00:00:00 2001 From: Chris Scott Date: Fri, 7 Nov 2025 12:19:42 +1300 Subject: [PATCH 1/7] initial globus compute docs --- .../Globus_Compute/index.md | 77 +++++++++++++++++++ 1 file changed, 77 insertions(+) create mode 100644 docs/Scientific_Computing/Globus_Compute/index.md diff --git a/docs/Scientific_Computing/Globus_Compute/index.md b/docs/Scientific_Computing/Globus_Compute/index.md new file mode 100644 index 000000000..bb6d795f4 --- /dev/null +++ b/docs/Scientific_Computing/Globus_Compute/index.md @@ -0,0 +1,77 @@ +# Globus Compute + +!!! warning + + Our Globus Compute offering is in early-access mode and may change. Please let us know if you have any feedback or suggestions for improvements. + +## Overview + +[Globus Compute](https://www.globus.org/compute) provides a secure, scalable way to run Python functions remotely on compute resources. +On Mahuika, we host a Globus Compute multi-user endpoint, allowing users to submit and manage compute tasks through +Globus or compatible clients without needing to connect to the HPC via SSH or OnDemand. +Globus Compute has [comprehensive documentation](https://globus-compute.readthedocs.io/en/latest/quickstart.html) and here we will just +highlight the specifics of working with the Mahuika endpoints. + +## Key concepts + +`Endpoint` + +: A Globus Compute service running on the HPC system that executes user functions + +`Single-user mode` + +: Each user runs and manages their own endpoint on a login node, which can execute their functions (this approach will not be discussed + here but is documented on the Globus Compute site) + +`Multi-user mode` + +: A centrally managed endpoint that all REANNZ HPC users can send tasks to; a user identity mapping process + maps the user's Globus identity to their account on Mahuika + +`Executors` + +: Manage job submissions to Slurm or execute jobs directly on login nodes + +`Authentication` + +: Handled via Globus Auth - users must have a Globus account, which they can sign into with institutional or Globus IDs, and they + must add a linked identity with the "NeSI Keycloak" to their Globus account + +## Requirements + +You must have a NeSI account, Globus account and have linked an identity from your Globus account to the NeSI Keycloak. This can be achieved by +navigating to the [NeSI HPC Storage](https://app.globus.org/file-manager?origin_id=763d50ee-e814-4080-878b-6a8be5cf7570) in the Globus +web app and ensuring you can see your NeSI files. + +## Endpoints + +| Name | Endpoint ID | Purpose | +|------------|----------------------------------------|--------------------------------------------------------------| +| nesi-login | `63c0b682-43d1-4b97-bf23-6a676dfdd8bd` | Lighweight tasks that are suitable to be run on a login node, such as submitting Slurm jobs, checking job status, etc. | +| nesi-slurm | `abf152c8-ad9b-453f-bcc8-3424284344f3` | Resource intensive tasks; work sent to this endpoint will run in a Slurm job | + +## `nesi-slurm` endpoint + +This endpoint submits work in Slurm jobs. The following configuration options are available via [`user_endpoint_config`](https://globus-compute.readthedocs.io/en/v2.20.1/reference/executor.html#globus_compute_sdk.Executor.user_endpoint_config): + +- `ACCOUNT_ID` (required): your NeSI project code +- `WALL_TIME` (optional, defaults to `00:05:00`): the wall time for the Slurm job that gets submitted (must be enough time for your function to complete) +- `MEM_PER_CPU` (optional, defaults to `2G`): amount of memory to be requested in the Slurm job +- `GPUS_PER_NODE` (optional, defaults to no GPU): request GPUs for the Slurm job + +## Limitation and known problems + +Limitations and known problems related to our current implementation are listed here. +If these are impacting your ability to use this service, please [let us know](mailto:support@nesi.org.nz). + +- Currently limited to a single CPU +- You must use Python 3.11 on the remote end + - we are exploring options to execute functions in containers, which could use different versions of Python +- You can only import Python packages that are available in the `Python/3.11.6-foss-2023a` environment module + - containerisation will help here too +- Globus Compute version 3.x + +## Other notes + +- Globus Compute uses token based auth after the initial setup (along with guest collections things can be fully automated) +- standard access and usage policies, quotas and accounting rules apply (active project, no compute intensive work on login endpoint, etc) From f5eed07b7335cba7fae02b1e4d66274b06bb42ae Mon Sep 17 00:00:00 2001 From: Chris Scott Date: Fri, 7 Nov 2025 12:22:01 +1300 Subject: [PATCH 2/7] fix bullets --- docs/Scientific_Computing/Globus_Compute/index.md | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/docs/Scientific_Computing/Globus_Compute/index.md b/docs/Scientific_Computing/Globus_Compute/index.md index bb6d795f4..d0203bb83 100644 --- a/docs/Scientific_Computing/Globus_Compute/index.md +++ b/docs/Scientific_Computing/Globus_Compute/index.md @@ -65,10 +65,8 @@ Limitations and known problems related to our current implementation are listed If these are impacting your ability to use this service, please [let us know](mailto:support@nesi.org.nz). - Currently limited to a single CPU -- You must use Python 3.11 on the remote end - - we are exploring options to execute functions in containers, which could use different versions of Python -- You can only import Python packages that are available in the `Python/3.11.6-foss-2023a` environment module - - containerisation will help here too +- You must use Python 3.11 (we are exploring options to execute functions in containers, which will enable use of different Python versions) +- You can only import Python packages that are available in the `Python/3.11.6-foss-2023a` environment module (containerisation will help here too) - Globus Compute version 3.x ## Other notes From a092a733b180ee996c56dcecafb12bd507f23b0c Mon Sep 17 00:00:00 2001 From: Chris Scott Date: Fri, 7 Nov 2025 12:33:46 +1300 Subject: [PATCH 3/7] add globus compute example --- .../Globus_Compute/index.md | 34 +++++++++++++++++++ 1 file changed, 34 insertions(+) diff --git a/docs/Scientific_Computing/Globus_Compute/index.md b/docs/Scientific_Computing/Globus_Compute/index.md index d0203bb83..02c128944 100644 --- a/docs/Scientific_Computing/Globus_Compute/index.md +++ b/docs/Scientific_Computing/Globus_Compute/index.md @@ -59,6 +59,39 @@ This endpoint submits work in Slurm jobs. The following configuration options ar - `MEM_PER_CPU` (optional, defaults to `2G`): amount of memory to be requested in the Slurm job - `GPUS_PER_NODE` (optional, defaults to no GPU): request GPUs for the Slurm job +## Simple example + +1. Install Python 3.11 and create a virtual environment + ``` + python -m venv venv + source venv/bin/activate + ``` +2. Install Globus Compute + ``` + pip install "globus_compute_sdk>=3,<4" + ``` +3. Create a simple Python script (replacing `` with your project code) + ``` + # test.py + from globus_compute_sdk import Executor + + def hello_from_node(): + import os + import getpass + return f"Hello, this function ran as {getpass.getuser()} on {os.uname().nodename}" + + mep_id = "abf152c8-ad9b-453f-bcc8-3424284344f3" + with Executor() as ex: + ex.endpoint_id = mep_id + ex.user_endpoint_config = {"ACCOUNT_ID": ""} + f = ex.submit(hello_from_node) + print(f.result()) + ``` +4. Run the test + ``` + python test.py + ``` + ## Limitation and known problems Limitations and known problems related to our current implementation are listed here. @@ -67,6 +100,7 @@ If these are impacting your ability to use this service, please [let us know](ma - Currently limited to a single CPU - You must use Python 3.11 (we are exploring options to execute functions in containers, which will enable use of different Python versions) - You can only import Python packages that are available in the `Python/3.11.6-foss-2023a` environment module (containerisation will help here too) +- There can be a lag of around 1 minute to run a function if you have not used the endpoint recently - Globus Compute version 3.x ## Other notes From 5a9091b24b74cd981f2e2d5e1d3f39aa57bebe6b Mon Sep 17 00:00:00 2001 From: Chris Scott Date: Fri, 7 Nov 2025 12:34:48 +1300 Subject: [PATCH 4/7] typo --- docs/Scientific_Computing/Globus_Compute/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/Scientific_Computing/Globus_Compute/index.md b/docs/Scientific_Computing/Globus_Compute/index.md index 02c128944..57d52b95f 100644 --- a/docs/Scientific_Computing/Globus_Compute/index.md +++ b/docs/Scientific_Computing/Globus_Compute/index.md @@ -92,7 +92,7 @@ This endpoint submits work in Slurm jobs. The following configuration options ar python test.py ``` -## Limitation and known problems +## Limitations and known problems Limitations and known problems related to our current implementation are listed here. If these are impacting your ability to use this service, please [let us know](mailto:support@nesi.org.nz). From 33941ec0386a3ccbea26c3f62e9863f1bf558ce1 Mon Sep 17 00:00:00 2001 From: Chris Scott Date: Mon, 24 Nov 2025 11:57:40 +1300 Subject: [PATCH 5/7] change some nesi -> reannz Signed-off-by: Chris Scott --- .../Scientific_Computing/Globus_Compute/index.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/docs/Scientific_Computing/Globus_Compute/index.md b/docs/Scientific_Computing/Globus_Compute/index.md index 57d52b95f..9d81ad5db 100644 --- a/docs/Scientific_Computing/Globus_Compute/index.md +++ b/docs/Scientific_Computing/Globus_Compute/index.md @@ -39,22 +39,22 @@ highlight the specifics of working with the Mahuika endpoints. ## Requirements -You must have a NeSI account, Globus account and have linked an identity from your Globus account to the NeSI Keycloak. This can be achieved by +You must have a REANNZ HPC account, Globus account and have linked an identity from your Globus account to the NeSI Keycloak. This can be achieved by navigating to the [NeSI HPC Storage](https://app.globus.org/file-manager?origin_id=763d50ee-e814-4080-878b-6a8be5cf7570) in the Globus -web app and ensuring you can see your NeSI files. +web app and ensuring you can see your REANNZ HPC files. ## Endpoints -| Name | Endpoint ID | Purpose | -|------------|----------------------------------------|--------------------------------------------------------------| -| nesi-login | `63c0b682-43d1-4b97-bf23-6a676dfdd8bd` | Lighweight tasks that are suitable to be run on a login node, such as submitting Slurm jobs, checking job status, etc. | -| nesi-slurm | `abf152c8-ad9b-453f-bcc8-3424284344f3` | Resource intensive tasks; work sent to this endpoint will run in a Slurm job | +| Name | Endpoint ID | Purpose | +|--------------|----------------------------------------|--------------------------------------------------------------| +| reannz-login | `63c0b682-43d1-4b97-bf23-6a676dfdd8bd` | Lighweight tasks that are suitable to be run on a login node, such as submitting Slurm jobs, checking job status, etc. | +| reannz-slurm | `abf152c8-ad9b-453f-bcc8-3424284344f3` | Resource intensive tasks; work sent to this endpoint will run in a Slurm job | -## `nesi-slurm` endpoint +## `reannz-slurm` endpoint This endpoint submits work in Slurm jobs. The following configuration options are available via [`user_endpoint_config`](https://globus-compute.readthedocs.io/en/v2.20.1/reference/executor.html#globus_compute_sdk.Executor.user_endpoint_config): -- `ACCOUNT_ID` (required): your NeSI project code +- `ACCOUNT_ID` (required): your REANNZ HPC project code - `WALL_TIME` (optional, defaults to `00:05:00`): the wall time for the Slurm job that gets submitted (must be enough time for your function to complete) - `MEM_PER_CPU` (optional, defaults to `2G`): amount of memory to be requested in the Slurm job - `GPUS_PER_NODE` (optional, defaults to no GPU): request GPUs for the Slurm job From 0f14624a676d670abdd79f34387a0e52709f6a6f Mon Sep 17 00:00:00 2001 From: Cal <35017184+CallumWalley@users.noreply.github.com> Date: Thu, 4 Dec 2025 14:11:09 +1300 Subject: [PATCH 6/7] Move to fix new nav structure. Signed-off-by: Cal <35017184+CallumWalley@users.noreply.github.com> --- .../Globus_Compute/index.md | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename docs/{Scientific_Computing => Batch_Computing}/Globus_Compute/index.md (100%) diff --git a/docs/Scientific_Computing/Globus_Compute/index.md b/docs/Batch_Computing/Globus_Compute/index.md similarity index 100% rename from docs/Scientific_Computing/Globus_Compute/index.md rename to docs/Batch_Computing/Globus_Compute/index.md From e7ece52d9ca2fc44662368c62c836b770f1b325b Mon Sep 17 00:00:00 2001 From: Cal <35017184+CallumWalley@users.noreply.github.com> Date: Thu, 4 Dec 2025 16:58:55 +1300 Subject: [PATCH 7/7] change category to page Signed-off-by: Cal <35017184+CallumWalley@users.noreply.github.com> --- docs/Batch_Computing/Globus_Compute/{index.md => .md} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename docs/Batch_Computing/Globus_Compute/{index.md => .md} (100%) diff --git a/docs/Batch_Computing/Globus_Compute/index.md b/docs/Batch_Computing/Globus_Compute/.md similarity index 100% rename from docs/Batch_Computing/Globus_Compute/index.md rename to docs/Batch_Computing/Globus_Compute/.md