Support CUDA 12.2

[jameslamb]

Description

• switches to CUDA 12.2.2 for building conda packages and wheels
• adds new tests running against CUDA 12.2.2

Notes for Reviewers

This is part of ongoing work to build and test packages against CUDA 12.2.2 across all of RAPIDS.

For more details see:

• Add support for CUDA 12.2 wheels build-planning#7
• add CUDA 12.2 conda and wheel jobs shared-workflows#166

Planning a second round of PRs to revert these references back to a proper branch-24.{nn} release branch of shared-workflows once rapidsai/shared-workflows#166 is merged.

I intentionally did not add a CUDA 12.2 environment for ANN benchmarks, as I assumed that would be more involved and because it isn't strictly necessary to support building and publishing packages that support CUDA 12.2.

raft/dependencies.yaml

Lines 23 to 26 in 93a504e

	bench_ann:
	output: conda
	matrix:
	cuda: ["11.8", "12.0"]

(created with rapids-reviser)

[bdice]

https://github.com/rapidsai/raft/actions/runs/7494925981/job/20403970360?pr=2092#step:7:644

-- CPM: Using local package CCCL@2.2.0.0

I suspect this is causing issues due to buggy install rules which we've patched in rapids-cmake but aren't present in the CUDA 12.2 local copy that this build is finding.

We probably need to do something to prevent RAPIDS from using the CUDA 12.2 CCCL version 2.2.0, since it doesn't have the CMake patches we need. Always downloading is the only solution I can think of, unless there's a safe way for us to force patching a local package. I think this probably needs something like CPM_DOWNLOAD_CCCL? cc: @robertmaynard @vyasr

[bdice]

It looks like force-downloading CCCL has fixed the build here. I opened a rapids-cmake PR to always download CCCL: rapidsai/rapids-cmake#522

[vyasr]

The diagnosis is right. I'm not sure of the fix though. I'll comment on the rapids-cmake PR.

[bdice]

While working on #2102, I noticed an issue. In the libraft-headers-only recipe, we currently rely on cuda-cudart-dev in host to add some desired run-exports for cuda-cudart. This is going to need some rethinking for CUDA 12.2, since the run-exports we've been using to our advantage in CUDA 12.0 are not going to be correct for CUDA Enhanced Compatibility for CUDA 12.2. You can see that here, where CI is pulling cuda-version=12.2 for the 12.0 CI job. https://github.com/rapidsai/raft/actions/runs/7562776933/job/20636164208?pr=2092#step:7:345 We're not testing what we thought we were testing (it's really 12.2 libraries and not 12.0 libraries).

We will also need to ignore the strong run export of cuda-version >={{ cuda_version }} from cuda-nvcc, in my understanding. https://github.com/conda-forge/cuda-nvcc-feedstock/blob/717380326f9e9599c77fac378b19adf894380b3c/recipe/meta.yaml#L47

I'm not sure if we want to treat this as a CUDA packaging bug or as a bug in libraft's recipe (it applies to all of RAPIDS). We had discussed "assuming CUDA Enhanced Compatibility by default" and considered the possibility of loosening the CUDA compiler pin to 12.* in conda-forge rather than migrating for each minor version. cc: @jakirkham

[jakirkham]

Looks like this is now green! 🥳

[jakirkham]

One of the jobs failed due to git clone. So needs a restart. All the other ones passed

[bdice]

All CI is passing and CI logs look good. We need an ops review from @AyodeAwe / @raydouglass to proceed. I'll trigger the /merge for now.

[bdice]

/merge