Instuctions need revision

[thomas-robinson]

The first step under Set up for all configurations says "Obtain the spack-stack container via s3 bucket /place/path/here". It looks like this is incomplete. I don't know where to get the container from.

There is an instruction that says "top-dir is the first dir in your $PWD."
Here is my $PWD

$ echo $PWD
/work/Thomas.Robinson/EPIC/modulefiles

Should I bind in /work/Thomas.Robinson/EPIC, /work/Thomas.Robinson, /work? I don't think I should bind in /.

I found a container on ursa that was similar: /scratch3/NCEPDEV/nems/role.epic/containers/ubuntu22.04-intel-ufs-env-v1.9.2.img
Maybe this container is old, because I followed the intel-sandbox directions and it did not find the -s option:

$ python3 convert-modules.py -i $img -o $PWD/spack-stack-1.9.2 -s ${PWD}/intel-sandbox
usage: convert-modules.py [-h] -i IMG -o OUTPUT_DIR
convert-modules.py: error: unrecognized arguments: -s /work/Thomas.Robinson/EPIC/modulefiles/intel-sandbox
$ python3 convert-modules.py --help
usage: convert-modules.py [-h] -i IMG -o OUTPUT_DIR

Modify Lua environment variable references

optional arguments:
  -h, --help            show this help message and exit
  -i IMG, --container-image IMG
                        Path to the singularity image file containing spack-
                        stack
  -o OUTPUT_DIR, --output-dir OUTPUT_DIR
                        Path to the output directory for modified Lua files

[thomas-robinson]

I was able to pull the container with wget https://noaa-ufs-gdas-pds.s3.amazonaws.com/spack-stack-containers/ubuntu22.04-intel-ufs-env-v1.9.2-runtime.img. I followed the sandbox directions. Which still results in the same error

convert-modules.py: error: unrecognized arguments: -s /work/Thomas.Robinson/EPIC/modulefiles/intel-sandbox
$ python3 convert-modules.py --help
usage: convert-modules.py [-h] -i IMG -o OUTPUT_DIR

Modify Lua environment variable references

optional arguments:
  -h, --help            show this help message and exit
  -i IMG, --container-image IMG
                        Path to the singularity image file containing spack-
                        stack
  -o OUTPUT_DIR, --output-dir OUTPUT_DIR
                        Path to the output directory for modified Lua files

It looks like the current main version of container-scripts has this option ( https://github.com/NOAA-EPIC/container-scripts/blob/develop/convert-modules.py#L176) but what is copied out of the container it older and does not have this version of convert-modules.py. convert-modules.py seems to rely on another file that is not present in the container if I try to run with the current version:

$ python3 convert-modules.py -i $img -o $PWD/spack-stack-1.9.2 -s $PWD/intel-sandbox
/usr/bin/cp: cannot stat '/opt/container-scripts/bind_directories.conf': No such file or directory
singularity exec -e -B /work /work/Thomas.Robinson/EPIC/ubuntu22.04-intel-ufs-env-v1.9.2.img cp -r /opt/spack-stack/spack-stack-1.9.2/envs/unified-env/install/modulefiles .
ls: cannot access './modulefiles/Core': No such file or directory
Traceback (most recent call last):
  File "/work5/ter/EPIC/modulefiles/convert-modules.py", line 227, in <module>
    compiler_type=stack_type.split("-")[1]
                  ~~~~~~~~~~~~~~~~~~~~~^^^
IndexError: list index out of range

To reproduce (assuming you have set SINGULARITY_CACHEDIR and SINGULARITY_TMPDIR):

wget https://noaa-ufs-gdas-pds.s3.amazonaws.com/spack-stack-containers/ubuntu22.04-intel-ufs-env-v1.9.2-runtime.img
img=... (path to image)
singularity exec -B /work $img cat /opt/container-scripts/convert-modules.py > convert-modules.py
singularity build --sandbox intel-sandbox docker://intel/hpckit:2024.2.0-1-devel-ubuntu22.04
python3 convert-modules.py -i $img -o $PWD/spack-stack-1.9.2 -s $PWD/intel-sandbox
git clone https://github.com/NOAA-EPIC/container-scripts.git
cp container-scripts/convert-modules.py .
python3 convert-modules.py -i $img -o $PWD/spack-stack-1.9.2 -s $PWD/intel-sandbox

[EdwardSnyder-NOAA]

I found the issue. Looks like your img variable is still pointing the container you found on Ursa and not the container (ubuntu22.04-intel-ufs-env-v1.9.2-runtime.img) from the s3 bucket. This is from your output, it prints the img variable the singularity command is using and it isn't looking at the runtime container:

$ python3 convert-modules.py -i $img -o $PWD/spack-stack-1.9.2 -s $PWD/intel-sandbox
/usr/bin/cp: cannot stat '/opt/container-scripts/bind_directories.conf': No such file or directory
singularity exec -e -B /work /work/Thomas.Robinson/EPIC/ubuntu22.04-intel-ufs-env-v1.9.2.img cp -r /opt/spack-stack/spack-stack-1.9.2/envs/unified-env/install/modulefiles .
ls: cannot access './modulefiles/Core': No such file or directory
Traceback (most recent call last):
File "/work5/ter/EPIC/modulefiles/convert-modules.py", line 227, in
compiler_type=stack_type.split("-")[1]
~~~~~~~~~~~~~~~~~~~~~^^^
IndexError: list index out of range

Setting img=/full/path/to/ubuntu22.04-intel-ufs-env-v1.9.2-runtime.img will resolve the issue.

In addition, the runtime container you have has a fork of the container-scripts repo, but virtually is the same as the main container-scripts repo. I'm still rebuilding the runtime container with main container-scripts repo though, and it might take an hour from now to complete. I would recommend to use the container you have until I can push the newly updated one to the s3 bucket.