support cuda 12, rerender build #29
Conversation
…nda-forge-pinning 2023.11.18.11.24.55
|
hey @jakirkham, I'm trying to resolve the cuda12 issue in this PR. (this PR both rerenders the build for our own testing on CI, and attempts to resolve the problems) I have two questions at the moment:
from the nvidia forum it seems to suggest that cuda wasn't installed setup correctly? |
|
perhaps relevant info, I also updated the conda_build_config in this PR, to try to match the ci_support files that were getting generated in conda-forge/cudadecon-feedstock#27 ... so might have made an error there |
|
Sorry for the delay Talley So It seems like CMake (IIUC where Know that only answers half of your question. Though let's start there and see how things go |
tlambert03
changed the title
|
thanks @jakirkham. Things look to be building well now on both cuda 11 and 12!
|
|
Huzzah! 🎉 Thanks Talley! 🙏 Do you happen to have the log where GCC 12 didn't work? Also curious would GCC 11 work? |
|
here was the log where I tried gcc12 with cuda 12: didn't try gcc11, but can! |
|
Thanks Talley! 🙏 Ah one other thing... Noted we are using Line 106 in e822f75 Could we instead use IOW making a change like this? - find_package(CUDA REQUIRED)
+ enable_language(CUDA)
+ find_package(CUDAToolkit REQUIRED) |
|
I actually did try to do that briefly in 6722213 it worked for linux, but failed to find |
|
Maybe I missed it, but was |
|
Nope! I'll try that |
That is insufficient, as
|
|
yeah, 6722213 had all of those changes ... (it didn't just use enable language). i just left out the call to |
|
It looks like adding find package cudatoolkit didn't resolve the missing complex.h issue on windows. Linux works fine though. @robertmaynard Any thoughts on what I might be missing? |
The Your failures with By linking to the In general when using C++ and the CUDA toolkit you want to use |
| export PATH="$PATH:$BUILD_PREFIX/nvvm/bin/" | ||
|
|
There was a problem hiding this comment.
Am curious if we still need this after the CMake changes
| export PATH="$PATH:$BUILD_PREFIX/nvvm/bin/" | |
| # todo: see if we can drop this and still find `cicc` | |
| # export PATH="$PATH:$BUILD_PREFIX/nvvm/bin/" | |
There was a problem hiding this comment.
The above changes won't fix this, as CMake isn't what is invoking cicc. This points to some issue with the env
I thought nvcc needs to be in the HOST not BUILD as it doesn't change when cross-compiling
There was a problem hiding this comment.
nvcc needs to be in requirements/build as it is a build tool
Independent of that nvcc still has some content that that could depend on the target architecture ( conda-forge/cuda-nvcc-impl-feedstock#4 )
We do add nvvm/bin to the path in nvcc.profile. Is it possible this path needs to be changed? If so, is there a good way to check what nvcc thinks this path is now?
There was a problem hiding this comment.
nvcc needs to be in requirements/build as it is a build tool
👍 I must be mixing things up. Looking at other projects the bigger difference is they are using {{ compiler('cuda') }} and not directly the cuda-nvcc package. That might the source of the issue
There was a problem hiding this comment.
All good. It can be confusing 😅
Yeah {{ compiler('cuda') }} is Jinja {{ }} with a special function compiler, which takes the str provided (in this case cuda) and appends _compiler to that to form the variant key (so cuda_compiler). It then uses this variant key to look for what cuda_compiler should be. There are a few places it will look for this variant key one being conda_build_config.yaml, which in this case specifies these options:
cudadecon/recipe/conda_build_config.yaml
Lines 16 to 18 in 9379425
nvcc refers to conda-forge/nvcc-feedstock, which we used for CUDA 11 builds. cuda-nvcc refers to conda-forge/cuda-nvcc-feedstock, which we have added for CUDA 12. Depending on whether a CUDA 11 or 12 build is done, it will pick from these
There is one last step where conda-build converts the cuda_compiler selected into the compiler used, which is it adds _{{ target_platform }} to it. IOW nvcc would become nvcc_linux-64 for builds targeting Linux x86_64 on CUDA 11. Or cuda-nvcc would become cuda-nvcc_linux-aarch64 for builds targeting Linux ARM 64-bit. The package installed will be selected to run on the build platform
Hopefully that makes more sense
|
thanks a lot for your time @robertmaynard, much appreciated. Yeah I switched the I'll reread your comments, and keep digging a bit. thanks again both! |
Happy to help. It looks like the |
|
Were you able to make more progress here Talley? No worries if not. Just wanted to check in 🙂 |
|
ahh, shoot :) dropped the ball again didn't I @jakirkham 😂 |
|
I'm seeing this too, and I think it's because |
|
thanks so much for commenting here @LourensVeen, I appreciate it. I see in that thread you said:
and
sorry to ask a basic question: but can can you clarify where you're adding those symlinks? That is, if I'd like to manually patch things so as to get this building on CI again, how might I directly modify my own |
…nda-forge-pinning 2024.07.15.13.31.46
|
@jakirkham, i've actually got one working windows build now! (on cuda12) 🎉 can you spot why it might not be working with cuda11.2? differences I notice between the working one and the non-working one:
(working one logs begin around here) |
|
since I'd ❤️❤️❤️❤️ to get this 🐒 of my 🔙 ... 😂 I'm gonna merge this as is, without support for cuda 11.2 on windows. will open a new PR |
|
Hooray!!! 🥳 🍾 Thanks Talley! 👏 Nice to see this reach completion 😄 Will comment in the other issues/PRs |
(Not sure if this is still relevant, but I thought I'd reply anyway, would love to know if you found another solution.) Ah, I haven't actually tried to patch things while building a package. I'm trying to renovate some old code, and I'm still trying to get it to build against conda-installed dependencies locally. So I made an env and activated it and patched it using $ ln -s ${CONDA_PREFIX}/bin/nvcc.profile ${CONDA_PREFIX}/targets/x86_64-linux/bin/nvcc.profile
$ ln -s ${CONDA_PREFIX}/bin/crt ${CONDA_PREFIX}/targets/x86_64-linux/bin/crtIn a build environment, I think you want to use |
|
thanks @LourensVeen. Yeah the final thing that did it for me seems like it was likely unrelated: I just needed to add in one of my nested CMake files. still don't know why I needed that only for windows, but it's all building now and i'm gonna move on with my life 😂 |
This updates stuff for CUDA 12.
one casualty is the
camcor.cufile which provides camera correction code (which probably doesn't need to be in this library anyway, but I could update it again if need be).cmakelists is simplified as well to take advantage of built in cuda support