Neko on JUPITER

[nick-morse]

To compile Neko on JUPITER

(For a general guide on how to build Neko for NVIDIA GPUs please see: #540)

• Make sure have to have the following modules loaded: ml GCC OpenMPI OpenBLAS CUDA
• Configure Neko using the following options, where OpenMP support with NEKO_GS_STRTGY=3 were found to be performant:

./configure FC=gfortran CC=gcc \
            MPIFC=mpifort MPICC=mpicc \
	        FCFLAGS="-O3" CFLAGS="-O3" \
	        LDFLAGS=-lstdc++ \
	        CUDA_ARCH="-arch sm_90" \
            NEKO_GS_STRTGY=3 \
 	        --with-cuda=/e/software/default/stages/2026/software/CUDA/13 \
	        --enable-device-mpi \
            --enable-openmp

• Build using make && make install -j12
• Set the following options in the shell/jobscript before running neko:

export OMP_NUM_THREADS="72"
export UCX_RNDV_THRESH=intra:131072,inter:131072 
export UCX_RNDV_SCHEME=put_zcopy 
export UCX_MAX_RNDV_RAILS=1 
export UCX_TLS=rc_x,self,sm,cuda_ipc,gdr_copy,cuda_copy

• This configuration produced a working installation on JUPITER. The job output is as follows: (note output of OpenMP threads per rank)

    _  __  ____  __ __  ____  
   / |/ / / __/ / //_/ / __ \ 
  /    / / _/  / ,<   / /_/ / 
 /_/|_/ /___/ /_/|_|  \____/  

 (version: 1.99.3)
 (build: 2026-03-05 on aarch64-unknown-linux-gnu using gnu)


 -------Job Information--------
 Start time: 07:35 / 2026-03-06
 Running on: 3072 MPI ranks, using 72 thrds each
 CPU type  : ARM Neoverse V2
 Bcknd type: Accelerator (CUDA)
 Dev. name : NVIDIA GH200 120GB