Add CI to compare stacks, checking for both modules and extensions

[ocaisa]

No description provided.

[eessi-bot]

Instance eessi-bot-mc-aws is configured to build for:

• architectures: x86_64/generic, x86_64/intel/haswell, x86_64/intel/sapphire_rapids, x86_64/intel/skylake_avx512, x86_64/amd/zen2, x86_64/amd/zen3, aarch64/generic, aarch64/neoverse_n1, aarch64/neoverse_v1
• repositories: eessi.io-2023.06-software, eessi.io-2023.06-compat

[eessi-bot]

Instance eessi-bot-mc-azure is configured to build for:

• architectures: x86_64/amd/zen4
• repositories: eessi.io-2023.06-software, eessi.io-2023.06-compat

[riscv-eessi-io-bot]

Instance eessi-bot-riscv is configured to build for:

• architectures: riscv64/generic
• repositories: riscv.eessi.io-20240402

[eessi-bot-trz42]

Instance trz42-GH200-jr is configured to build for:

• architectures: aarch64/nvidia/grace
• repositories: eessi.io-2023.06-software

[eessi-bot-casparvl-eessi]

Instance eessi-bot-casparvl is configured to build for:

• architectures: x86_64/amd/zen4, x86_64/amd/zen2
• repositories: eessi.io-2023.06-software, eessi-hpc.org-2023.06-compat, eessi-hpc.org-2023.06-software, eessi.io-2023.06-compat

[gpu-bot-ugent]

Instance eessi-bot-vsc-ugent is configured to build for:

• architectures: x86_64/amd/zen3
• repositories: eessi-hpc.org-2023.06-compat, eessi.io-2023.06-software, eessi-hpc.org-2023.06-software, eessi.io-2023.06-compat

[ocaisa]

@casparvl This highlights the remaining differences between Zen4 and the rest of the software stacks:

Comparing /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/amd/zen4/modules/all to /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/generic/modules/all
{
  "module_differences": {
    "missing": [
      "GDRCopy/2.3.1-GCCcore-12.3.0.lua",
      "ipympl/0.9.3-foss-2023a.lua",
      "UCX-CUDA/1.14.1-GCCcore-12.3.0-CUDA-12.1.1.lua",
      "Z3/4.12.2-GCCcore-12.3.0-Python-3.11.3.lua",
      "NCCL/2.18.3-GCCcore-12.3.0-CUDA-12.1.1.lua",
      "OSU-Micro-Benchmarks/7.2-gompi-2023a-CUDA-12.1.1.lua",
      "CUDA-Samples/12.1-GCC-12.3.0-CUDA-12.1.1.lua",
      "CUDA/12.1.1.lua",
      "UCC-CUDA/1.2.0-GCCcore-12.3.0-CUDA-12.1.1.lua"
    ],
    "added": []
  },
  "extension_differences": [
    {
      "R-bundle-CRAN/2023.12-foss-2023a.lua": {
        "missing": [],
        "added": [
          "cmna/1.0.5",
          "XBRL/0.99.19.1",
          "rhandsontable/0.3.8"
        ]
      }
    }
  ]
}

[ocaisa]

Given that our plan is to remove the CUDA stuff from the CPU stacks, I could add a filter for modules containing CUDA in the name or version

[casparvl]

Given that our plan is to remove the CUDA stuff from the CPU stacks, I could add a filter for modules containing CUDA in the name or version

Maybe. Or maybe this should be a good trigger to actually remove it...

[casparvl]

Ok, discussed with @bedroge and we'll deprecate first. See https://gitlab.com/eessi/support/-/issues/128 . So, if we want this check in CI, we'll have to make an exception at least for now for the CUDA stuff.

[ocaisa]

After syncing the R-bundle via #914 , the remaining gaps are the GPU-related modules and ipympl (https://github.com/EESSI/software-layer/actions/runs/13284436473/job/37528465316?pr=913#step:4:47)

[ocaisa]

@bedroge The missing ipympl is tricky as that should actually be removed from the others (see #587)

[ocaisa]

Maybe this should actually be moved to the docs repo? We already have a missing installations check here.

This check is a bit more strict though since it also checks the extensions.

[boegel]

Maybe this should actually be moved to the docs repo? We already have a missing installations check here.

This check is a bit more strict though since it also checks the extensions.

I think it's a better fit here, since it's a check for the installations we have in the software layer?

[ocaisa]

@bedroge with the merge of #962, cleaning out the modules for the GPU stuff from the CPU stacks should allow us this new CI to pass.

[bedroge]

The modules have been removed, so I'm rerunning the CI of this PR.

[eessi-bot]

PR merged! Moved [] to /project/def-users/SHARED/trash_bin/EESSI/software-layer/2025.03.04

[eessi-bot]

PR merged! Moved [] to /project/def-users/SHARED/trash_bin/EESSI/software-layer/2025.03.04