Virtual cuda

[haampie]

With this PR there are 3 packages providing CUDA: cuda-toolkit, nvhpc and nvhpc-slim.

1. cuda-toolkit: originally the cuda packages, and it just "forwards" cuda
   versions
2. nvhpc: provides multiple cuda's as a version list, corresponding to
   those from cuda-toolkit based on matching version.{json,txt}
3. nvhpc-slim: same as nvhpc, except it only provides a single "latest"
   cuda to reduce bandwidth & storage, and makes it easier to work with
   compat bounds like cuda@11.2:

This PR basically combines #29550 and #29155 (but provide cuda unconditionally) and #29742 and fixes #19365 for the most part.

As an example:

$ spack spec spfft +cuda ^cuda@:11.4 ^cuda-toolkit
    ^cuda-toolkit@11.4.4%gcc@11.2.0~allow-unsupported-compilers~dev arch=linux-ubuntu20.04-zen2

$ spack spec spfft +cuda ^cuda@:11.4 ^nvhpc-slim
    ^nvhpc-slim@21.9%gcc@11.2.0+blas+lapack~mpi install_type=single arch=linux-ubuntu20.04-zen2

$ spack spec spfft +cuda ^cuda@:11.4 ^nvhpc
    ^nvhpc@22.3%gcc@11.2.0+blas+lapack~mpi install_type=single arch=linux-ubuntu20.04-zen2

To me concretization makes sense for cuda-toolkit and nvhpc-slim. For nvhpc, I'm not entirely convinced this behavior is great, it picks up the newest nvhpc because it happens to provide an old 10.x cuda toolkit. But the build system likely doesn't know how to pick it up, so maybe some work has to be done to get the correct nvcc in the PATH (see e.g. https://cmake.org/cmake/help/latest/module/FindCUDAToolkit.html#search-behavior).

[haampie]

@spackbot run pipeline

[spackbot-app]

I've started that pipeline for you!

[fspiga]

I am drafting a long comment to this PR. Stay tuned.

[haampie]

Maybe it's better to have the discussion in #19365, since this is "just" one way to close it (and nobody is subscribed here).

[samcmill]

Adding the new nvhpc-slim package is only indirectly related to virtualizing the CUDA dependency. I would suggest splitting that out into a separate PR.

Regarding nvhpc-slim, how do you see package maintainers setting their dependencies on the NVIDIA compilers? Would they use nvhpc, or nvhpc-slim, would they be expected to support both?

[vkallesh]

Below message was seen when I tried to execute following command:

--snippet--

9 CMake Error at /home/amd/spack/opt/spack/linux-ubuntu20.04-zen2/gcc-11.2.0/
cmake-3.22.3-dvpfzzxzvjfzi67jmz6qfyi723bced5j/share/cmake-3.22/Modules/CMak
eDetermineCUDACompiler.cmake:179 (message):
10 Failed to find nvcc.
11
12 Compiler requires the CUDA toolkit. Please set the CUDAToolkit_ROOT
13 variable.
--snippet--

Command executed:
spack install spfft +cuda ^cuda@:11.4 ^nvhpc@22.3%gcc@11.2.0+blas+lapack~mpi install_type=single arch=linux-ubuntu20.04-zen2

Please guide me how to resolve above issue.

[wyphan]

@vkallesh Sorry for the late response, but would you mind creating a separate issue for this? You can pick "Build error" for the issue type, and the affected package is spfft.

[wyphan]

Per @haampie 's suggestion, I've taken over this PR with #30748.