Adding cuda-nvcc recipe #21350

adibbley · 2022-11-28T22:39:10Z

Checklist

conda-forge-linter · 2022-11-28T22:39:29Z

Hi! This is the friendly automated conda-forge-linting service.

I wanted to let you know that I linted all conda-recipes in your PR (recipes/cuda_nvcc) and found some lint.

Here's what I've got...

For recipes/cuda_nvcc:

The license item is expected in the about section.
The summary item is expected in the about section.
The recipe must have some tests.
Selectors are suggested to take a <two spaces>#<one space>[<expression>] form. See lines [2, 3, 4, 5, 13, 14, 15, 16]

conda-forge-linter · 2022-11-28T22:51:15Z

Hi! This is the friendly automated conda-forge-linting service.

I wanted to let you know that I linted all conda-recipes in your PR (recipes/cuda_nvcc) and found some lint.

Here's what I've got...

For recipes/cuda_nvcc:

The license item is expected in the about section.
The recipe must have some tests.

conda-forge-linter · 2022-11-29T14:19:50Z

Hi! This is the friendly automated conda-forge-linting service.

I wanted to let you know that I linted all conda-recipes in your PR (recipes/cuda_nvcc) and found some lint.

Here's what I've got...

For recipes/cuda_nvcc:

The top level meta keys are in an unexpected order. Expecting ['package', 'source', 'build', 'test', 'outputs', 'about', 'extra'].
The license item is expected in the about section.

conda-forge-linter · 2022-11-29T14:22:32Z

Hi! This is the friendly automated conda-forge-linting service.

I wanted to let you know that I linted all conda-recipes in your PR (recipes/cuda_nvcc) and found some lint.

Here's what I've got...

For recipes/cuda_nvcc:

The license item is expected in the about section.

recipes/cuda_nvcc/meta.yaml

Co-authored-by: Marius van Niekerk <marius.v.niekerk@gmail.com>

conda-forge-linter · 2022-11-30T18:14:03Z

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipes/cuda_nvcc) and found it was in an excellent condition.

recipes/cuda_nvcc/meta.yaml

conda-forge-linter · 2022-12-01T14:34:15Z

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipes/cuda-nvcc) and found it was in an excellent condition.

recipes/cuda-nvcc/nvcc.profile.patch

m3vaz · 2022-12-08T20:13:21Z

Doesn't nvcc have a maximum supported compiler version? Is that limit monitored here?

conda-forge-webservices · 2023-03-24T13:47:28Z

Hi! This is the friendly automated conda-forge-linting service.

I wanted to let you know that I linted all conda-recipes in your PR (recipes/cuda-nvcc) and found some lint.

Here's what I've got...

For recipes/cuda-nvcc:

Selectors are suggested to take a <two spaces>#<one space>[<expression>] form. See lines [21]

conda-forge-webservices · 2023-03-24T13:50:18Z

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipes/cuda-nvcc) and found it was in an excellent condition.

kkraus14 · 2023-03-27T21:34:56Z

recipes/cuda-nvcc/meta.yaml

+      - nvvm     # [linux]
+      - Library\nvvm  # [win]


Does it make sense for nvvm to be a separate package? I'm thinking about downstream packages like Numba who don't use nvcc but do use nvvm.

Regardless, this could be split out later if desired.

I think this could make sense. If we can raise this as an issue on the feedstock we could align this change for all formats, rather than a conda-forge only change here.

- add cuda-version - explicit target_names - about sections in every output - add arm-variant

recipes/cuda-nvcc/meta.yaml

kkraus14 · 2023-04-05T19:52:02Z

recipes/cuda-nvcc/meta.yaml

+        - cuda-nvcc_{{ target_platform }} =={{ version }}
+        - {{ pin_compatible("cuda-version", max_pin="x.x") }}
+      run_constrained:
+        - gcc_impl_{{ target_platform }} >=6,<13  # [linux]


Do we need to do similar with clang or msvc?

Since we only use clang on macOS and that is skipped, think we don't need to worry about that.

On Windows we don't distribute compiler packages and instead rely on CI to have them. Not sure what more we can do there

Since we only use clang on macOS and that is skipped, think we don't need to worry about that.

A user could presumably install it and want to use it in their environment though?

Sorry am not following. How would they install it? This recipe isn't built on macOS

There's linux-64 clang builds that someone could install and use in their environment.

Ah ok. That makes more sense. Thanks Keith! 🙏

There is a clang package. Though the package that encodes target_platform would be in clang-compiler-activation-feedstock. However this is only configured for macOS targets currently.

That said, suppose one could add Linux targets for clang. Maybe we could save this as a follow up issue (for whenever those packages get added)?

We can always repodata patch at that time. I'm fine to defer it to a follow up.

recipes/cuda-nvcc/nvcc.profile.patch

vyasr · 2023-04-11T23:35:38Z

recipes/cuda-nvcc/bld.bat

@@ -0,0 +1,17 @@
+if not exist %PREFIX% mkdir %PREFIX%


I'm unfamiliar with Windows, is this something we should be doing in all of our recipes? We didn't do that in libcublas, for instance. We're not specifically writing anything to the %LIBRARY_PREFIX% directory in this file.

vyasr · 2023-04-11T23:40:20Z

recipes/cuda-nvcc/build.sh

+            done
+        fi
+    else
+        cp -r $i ${PREFIX}


Some questions:

I assume that we're intentionally not including the contents of bin into the targetsDir?

We probably want to put the nvvm contents into the targetsDir right?

cicc is inside nvvm/bin, but nvcc needs to be able to find it. Does that need special handling to be placed into ${PREFIX}/bin?

The current layout is fine. The bin and nvvm directories will be copied into ${PREFIX}/bin/ and ${PREFIX}/nvvm. Since the relative layot has stayed consistent, nvcc doesn't need to be modified to know where the nvvm directory is

Are we OK writing to ${PREFIX}/nvvm? My understanding was that we were avoiding writing to such top-level directories as much as possible to avoid clobbering, but perhaps we're OK treating nvvm as CUDA-specific in the same way that we are treating targets as CUDA-specific.

Do we still want to include some of these contents in targets or only put them in the top-level ${PREFIX}/* directories?

We need nvvm in $PREFIX due to the relative pathing Robert mentioned. nvcc expects bin nvvm and targets to all be at the same level.

Clobbering shouldn't be a concern here as it was with things like src/example.c

We are already patching nvcc.profile, so it's not a big deal to patch a couple more lines.
If we are going with filesystem hierarchy, they should not be in $PREFIX and rather in somewhere like $PREFIX/lib/nvcc/nvvm and $PREFIX/lib/nvcc/targets just like gcc does with $PREFIX/lib/gcc

Thoughts @mbargull, @conda-forge/core?

nvcc.profile is not patched on Linux. It is only patched on Windows to address shipping different CCCL versions as requested in PR ( #21953 ). Preferably it would not be patched at all (even on Windows). Open to a better solution that doesn't require the patch on Windows (if we have ideas). Though this is probably best taken up in the feedstock since we are blocked on Windows builds atm ( #21723 (comment) ).

It's worth noting that there are already cases where we install non-traditional things in $PREFIX. For example go adds $PREFIX/go. With go, trying to move away from that model breaks the compiler in weird ways.

Typically all CUDA components are placed in the same relative paths where they are then QA’d and deployed. The proposal here suggests dividing them up between different paths that will deviate from the process. This risks introducing new issues unique to this non-standard deployment model and makes it harder to support.

Finally would add Numba is dependent on this location. So changing it here means Numba (and maybe other libraries) will need updates.

vyasr · 2023-04-11T23:47:10Z

recipes/cuda-nvcc/meta.yaml

+        - sysroot_{{ target_platform }} 2.17   # [linux]
+        - __glibc >=2.17                       # [linux]


Out of scope for fixing right now, but for future note: Is there no relationship between these packages? Should there be?

vyasr · 2023-04-11T23:49:58Z

recipes/cuda-nvcc/meta.yaml

+      host:
+        - sysroot_{{ target_platform }} 2.17   # [linux]
+        - __glibc >=2.17                       # [linux]
+        - cuda-version {{ cuda_version }}


What is the benefit of including cuda-version in the host environment? Isn't the purpose to be a metapackage ensuring compatibility across dependencies, i.e. something we really only need in run? What build requirements are being constrained by including this in host?

Defining this in host allows us to use the pin_compatible function in the run section. There may be more context though.

vyasr · 2023-04-11T23:50:21Z

recipes/cuda-nvcc/meta.yaml

+      run:
+        - sysroot_{{ target_platform }} 2.17   # [linux]
+        - __glibc >=2.17                       # [linux]
+        - cuda-cudart-dev


Do we need a version constraint here, or is that all managed using cuda-version?

I believe cuda-version suffices here.

recipes/cuda-nvcc/meta.yaml

Fix cuda-nvcc tests

Make a single output recipe.

jakirkham · 2023-04-13T20:27:38Z

We were running into some issues during testing, which we were struggling to resolve here. So for now we have cut this down to the minimal working recipe. Think it will be easier to try to add back the removed functionality in the feedstock. Though please let us know if you have other thoughts on that.

jakirkham · 2023-04-15T01:37:54Z

Planning on merging EOD Monday if no comments

robertmaynard

Testing the package locally ( conda install -c ..../staged-recipes/build_artifacts/ cuda-nvcc cuda-cudart ) in a clean env.

This creates a broken compiler, due to ${CXX} having no value. We need to either require a CXX compiler or only optionally add this env var when CXX exists.

(ctk) rmaynard@rmaynard-dt:~/Work/cmake/conda_test_build$ conda list
# packages in environment at /home/rmaynard/miniconda3/envs/ctk:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                       2_gnu    conda-forge
cuda-cccl_linux-64        12.0.90              ha770c72_1    conda-forge
cuda-cudart               12.0.107             hcb278e6_0    file:///home/rmaynard/Work/conda-forge/staged-recipes/build_artifacts
cuda-cudart-dev           12.0.107             hcb278e6_0    file:///home/rmaynard/Work/conda-forge/staged-recipes/build_artifacts
cuda-cudart-dev_linux-64  12.0.107             hcb278e6_0    file:///home/rmaynard/Work/conda-forge/staged-recipes/build_artifacts
cuda-cudart-static        12.0.107             hcb278e6_0    file:///home/rmaynard/Work/conda-forge/staged-recipes/build_artifacts
cuda-cudart-static_linux-64 12.0.107             hcb278e6_0    file:///home/rmaynard/Work/conda-forge/staged-recipes/build_artifacts
cuda-nvcc                 12.0.76              h59595ed_0    file:///home/rmaynard/Work/conda-forge/staged-recipes/build_artifacts
cuda-version              12.0                 hffde075_0    conda-forge
libgcc-ng                 12.2.0              h65d4601_19    conda-forge
libgomp                   12.2.0              h65d4601_19    conda-forge
libstdcxx-ng              12.2.0              h46fd767_19    conda-forge

jakirkham · 2023-04-18T01:10:06Z

Alex added a clearer error message above. If we want to do more here, let's raise an issue in the feedstock and discuss

jakirkham · 2023-04-18T01:11:41Z

Thanks all! 🙏

Let's follow up on anything else in the feedstock 🙂

Will also include a PR link related to follow ups pertaining to comment ( #21350 (comment) ) once available. ( edit: filed issue conda-forge/cuda-nvcc-feedstock#1 )

Adding cuda_nvcc recipe

6a9d375

adibbley added 3 commits November 28, 2022 17:46

Add license_file

5919ec4

Adding summary

5bd1429

Add suggested space

09e2531

Set correct extension for Windows source

dbd0844

Move tests block to expected order

7c85b37

nvcc.profile.patch is unnecessary on Windows

4ac9038

adibbley mentioned this pull request Nov 30, 2022

Build order for CUDA SDK #21382

Closed

49 tasks

mariusvniekerk reviewed Nov 30, 2022

View reviewed changes

recipes/cuda_nvcc/meta.yaml Outdated Show resolved Hide resolved

beckermr reviewed Nov 30, 2022

View reviewed changes

recipes/cuda_nvcc/meta.yaml Outdated Show resolved Hide resolved

Update recipes/cuda_nvcc/meta.yaml

d316b98

Co-authored-by: Marius van Niekerk <marius.v.niekerk@gmail.com>

beckermr reviewed Nov 30, 2022

View reviewed changes

recipes/cuda_nvcc/meta.yaml Outdated Show resolved Hide resolved

Removing output section and renaming to cuda-nvcc

417a5b7

jakirkham reviewed Nov 30, 2022

View reviewed changes

recipes/cuda_nvcc/meta.yaml Outdated Show resolved Hide resolved

adibbley added 2 commits December 1, 2022 09:25

Remove comment from example recipe

7924319

Rename recipe folder to cuda-nvcc

3d4b217

adibbley added 2 commits December 1, 2022 10:54

Remove extra directory from windows test

346d69d

Replace existence test with version check

bc8f875

isuruf reviewed Dec 1, 2022

View reviewed changes

recipes/cuda-nvcc/nvcc.profile.patch Outdated Show resolved Hide resolved

jakirkham changed the title ~~Adding cuda_nvcc recipe~~ Adding cuda-nvcc recipe Dec 5, 2022

jakirkham mentioned this pull request Dec 10, 2022

cross compile cuda support (cnt'd) conda-forge/conda-forge-ci-setup-feedstock#210

Merged

adibbley added 2 commits December 12, 2022 09:44

Updating for 12.0

4159367

Moving lib and include to targets

8f8850e

Windows only patch to support cuda-cccl

1271c83

Missing space for linter

e2397dc

adibbley mentioned this pull request Mar 24, 2023

Add libcublas recipe #21901

Merged

10 tasks

Merge branch 'main' into cuda-nvcc

786c01b

kkraus14 reviewed Mar 27, 2023

View reviewed changes

Update to match other component changes

46d478c

- add cuda-version - explicit target_names - about sections in every output - add arm-variant

kkraus14 reviewed Apr 5, 2023

View reviewed changes

Remove test/requires, and sysroot to build

f6aa345

kkraus14 approved these changes Apr 6, 2023

View reviewed changes

robertmaynard reviewed Apr 7, 2023

View reviewed changes

recipes/cuda-nvcc/nvcc.profile.patch Show resolved Hide resolved

vyasr suggested changes Apr 11, 2023

View reviewed changes

Move tests to top level.

c925141

bdice reviewed Apr 11, 2023

View reviewed changes

recipes/cuda-nvcc/meta.yaml Outdated Show resolved Hide resolved

adibbley and others added 5 commits April 12, 2023 09:19

Merge pull request #6 from bdice/fix-nvcc-tests

aaf8b3f

Fix cuda-nvcc tests

Merge branch 'main' into cuda-nvcc

054d71f

Merge branch 'main' into cuda-nvcc

25209da

Make a single output recipe.

7d0a96e

Merge pull request #7 from bdice/nvcc-single-output

4e8d8c9

Make a single output recipe.

kkraus14 approved these changes Apr 15, 2023

View reviewed changes

robertmaynard suggested changes Apr 17, 2023

View reviewed changes

Add error when CXX not set

579e8de

jakirkham merged commit 1bb91e4 into conda-forge:main Apr 18, 2023

jakirkham mentioned this pull request Apr 18, 2023

Add cuda-nvcc_{{ target_platform }} conda-forge/cuda-nvcc-feedstock#1

Closed

adibbley mentioned this pull request Aug 3, 2023

Create outputs for nvvm and crt conda-forge/cuda-nvcc-impl-feedstock#4

Closed

		- sysroot_{{ target_platform }} 2.17 # [linux]
		- __glibc >=2.17 # [linux]

Adding cuda-nvcc recipe #21350

Adding cuda-nvcc recipe #21350

Conversation

adibbley commented Nov 28, 2022 • edited Loading

conda-forge-linter commented Nov 28, 2022

conda-forge-linter commented Nov 28, 2022

conda-forge-linter commented Nov 29, 2022

conda-forge-linter commented Nov 29, 2022

conda-forge-linter commented Nov 30, 2022

conda-forge-linter commented Dec 1, 2022

m3vaz commented Dec 8, 2022

conda-forge-webservices bot commented Mar 24, 2023

conda-forge-webservices bot commented Mar 24, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jakirkham commented Apr 13, 2023

jakirkham commented Apr 15, 2023

robertmaynard left a comment

Choose a reason for hiding this comment

jakirkham commented Apr 18, 2023

jakirkham commented Apr 18, 2023 • edited Loading

adibbley commented Nov 28, 2022 •

edited

Loading

jakirkham commented Apr 18, 2023 •

edited

Loading