[SYCL] Use CUDA_PATH as a candidate for path discovery

[jchlanda]

Fixes: #5028

[mdtoguchi]

LGTM - does it make sense to add a test to be sure that the proper path is used when set?

[jchlanda]

LGTM - does it make sense to add a test to be sure that the proper path is used when set?

I wanted to add a test at the time of writing this patch, but this is such an esoteric case that I failed to come up with anything. I don't know how we could re-create the condition in the testing environment.

I'm tempted to say that this is also small/simple enough to maybe go without one?

[bader]

It's quite simple. Just set this environment variable for driver invocation, dump commands with -### option and check that variable value is used for include paths.
Here is an example of such test - https://github.com/intel/llvm/blob/sycl/clang/test/Driver/ve-toolchain.cpp#L85-L99.

[jchlanda]

It's quite simple. Just set this environment variable for driver invocation, dump commands with -### option and check that variable value is used for include paths. Here is an example of such test - https://github.com/intel/llvm/blob/sycl/clang/test/Driver/ve-toolchain.cpp#L85-L99.

I don't think that would work. Or at least not in the situation where this bug originally manifested. Clang would still fail to find the CUDA installation and fail the compilation, even though -### was appended. And since CUDA_PATH would be set to something invalid it would not get pulled into include paths. Please see attached:

> set CUDA_PATH="------------------>/WOOOF/------------------->"

> D:\Jakub\Tickets\5028>C:\Users\oneapi-core\user_files\jakub\llvm\build\bin\clang-cl.exe -### /TP  -fsycl -fsycl-unnamed-lambda /EHsc -sycl-std=2020 -fsycl-targets=nvptx64-nvidia-cuda -std:c++17 -c main.cpp


clang version 14.0.0 (https://github.com/intel/llvm.git fd0b108278e76a3f30947baeea45fc33510b1056)
Target: x86_64-pc-windows-msvc
Thread model: posix
InstalledDir: C:\Users\oneapi-core\user_files\jakub\llvm\build\bin
clang-cl: error: cannot find libdevice for sm_50; provide path to different CUDA installation via '--cuda-path', or pass '-nocudalib' to build without linking with libdevice
 "C:\\Users\\oneapi-core\\user_files\\jakub\\llvm\\build\\bin\\clang-cl.exe" "-cc1" "-triple" "nvptx64-nvidia-cuda" "-aux-triple" "x86_64-pc-windows-msvc" "-fsycl-is-device" "-fdeclare-spirv-builtins" "-fms-extensions" "-fms-compatibility" "-fdelayed-template-parsing" "-fms-compatibility-version=19.29.30136" "-fsycl-int-header=C:\\Users\\ONEAPI~1\\AppData\\Local\\Temp\\main-header-ff91d5.h" "-fsycl-int-footer=C:\\Users\\ONEAPI~1\\AppData\\Local\\Temp\\main-footer-c0e142.h" "-sycl-std=2020" "-fsycl-std-layout-kernel-params" "-fsycl-unique-prefix=9aa8db8a6d5ca1b1" "-emit-llvm-bc" "-emit-llvm-uselists" "-disable-free" "-clear-ast-before-backend" "-main-file-name" "main.cpp" "-mrelocation-model" "static" "-mframe-pointer=all" "-relaxed-aliasing" "-ffp-contract=on" "-fno-rounding-math" "-fno-verbose-asm" "-no-integrated-as" "-aux-target-cpu" "x86-64" "-internal-isystem" "C:\\Users\\oneapi-core\\user_files\\jakub\\llvm\\build\\bin\\..\\include\\sycl" "-internal-isystem" "C:\\Users\\oneapi-core\\user_files\\jakub\\llvm\\build\\bin\\..\\include" "-mlink-builtin-bitcode" "C:\\Users\\oneapi-core\\user_files\\jakub\\llvm\\build\\lib\\clang\\14.0.0\\../../clc\\remangled-l32-signed_char.libspirv-nvptx64--nvidiacl.bc" "-target-cpu" "sm_50" "-D_MT" "-flto-visibility-public-std" "--dependent-lib=libcmt" "--dependent-lib=oldnames" "-fexternc-nounwind" "-fdiagnostics-format" "msvc" "-debugger-tuning=gdb" "-fno-dwarf-directory-asm" "-resource-dir" "C:\\Users\\oneapi-core\\user_files\\jakub\\llvm\\build\\lib\\clang\\14.0.0" "-internal-isystem" "C:\\Users\\oneapi-core\\user_files\\jakub\\llvm\\build\\bin\\..\\include\\sycl" "-internal-isystem" "C:\\Users\\oneapi-core\\user_files\\jakub\\llvm\\build\\bin\\..\\include" "-internal-isystem" "C:\\Users\\oneapi-core\\user_files\\jakub\\llvm\\build\\lib\\clang\\14.0.0\\include" "-internal-isystem" "C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\Community\\VC\\Tools\\MSVC\\14.29.30133\\ATLMFC\\include" "-internal-isystem" "C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\Community\\VC\\Tools\\MSVC\\14.29.30133\\include" "-internal-isystem" "C:\\Program Files (x86)\\Windows Kits\\10\\include\\10.0.19041.0\\ucrt" "-internal-isystem" "C:\\Program Files (x86)\\Windows Kits\\10\\include\\10.0.19041.0\\shared" "-internal-isystem" "C:\\Program Files (x86)\\Windows Kits\\10\\include\\10.0.19041.0\\um" "-internal-isystem" "C:\\Program Files (x86)\\Windows Kits\\10\\include\\10.0.19041.0\\winrt" "-internal-isystem" "C:\\Program Files (x86)\\Windows Kits\\10\\include\\10.0.19041.0\\cppwinrt" "-internal-isystem" "C:\\Users\\oneapi-core\\user_files\\jakub\\llvm\\build\\lib\\clang\\14.0.0\\include" "-internal-isystem" "C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\Community\\VC\\Tools\\MSVC\\14.29.30133\\ATLMFC\\include" "-internal-isystem" "C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\Community\\VC\\Tools\\MSVC\\14.29.30133\\include" "-internal-isystem" "C:\\Program Files (x86)\\Windows Kits\\10\\include\\10.0.19041.0\\ucrt" "-internal-isystem" "C:\\Program Files (x86)\\Windows Kits\\10\\include\\10.0.19041.0\\shared" "-internal-isystem" "C:\\Program Files (x86)\\Windows Kits\\10\\include\\10.0.19041.0\\um" "-internal-isystem" "C:\\Program Files (x86)\\Windows Kits\\10\\include\\10.0.19041.0\\winrt" "-internal-isystem" "C:\\Program Files (x86)\\Windows Kits\\10\\include\\10.0.19041.0\\cppwinrt" "-fdeprecated-macro" "-fdebug-compilation-dir=D:\\Jakub\\Tickets\\5028" "-ferror-limit" "19" "-fms-extensions" "-fms-compatibility" "-fms-compatibility-version=19.29.30136" "-std=c++17" "-fdelayed-template-parsing" "-o" "C:\\Users\\ONEAPI~1\\AppData\\Local\\Temp\\main-cbef33\\main-sm_50.bc" "-x" "c++" "main.cpp"
 "C:\\Users\\oneapi-core\\user_files\\jakub\\llvm\\build\\bin\\append-file" "main.cpp" "--append=C:\\Users\\ONEAPI~1\\AppData\\Local\\Temp\\main-footer-c0e142.h" "--orig-filename=main.cpp" "--output=C:\\Users\\ONEAPI~1\\AppData\\Local\\Temp\\main-e63d78.cpp" "--use-include"
 "C:\\Users\\oneapi-core\\user_files\\jakub\\llvm\\build\\bin\\clang-cl.exe" "-cc1" "-triple" "x86_64-pc-windows-msvc19.29.30136" "-sycl-std=2020" "-fsycl-std-layout-kernel-params" "-fsycl-unique-prefix=9aa8db8a6d5ca1b1" "-include" "C:\\Users\\ONEAPI~1\\AppData\\Local\\Temp\\main-header-ff91d5.h" "-dependency-filter" "C:\\Users\\ONEAPI~1\\AppData\\Local\\Temp\\main-header-ff91d5.h" "-fsycl-enable-int-header-diags" "-fsycl-is-host" "-emit-obj" "-mrelax-all" "-mincremental-linker-compatible" "--mrelax-relocations" "-disable-free" "-clear-ast-before-backend" "-main-file-name" "main.cpp" "-mrelocation-model" "pic" "-pic-level" "2" "-mframe-pointer=none" "-relaxed-aliasing" "-fmath-errno" "-ffp-contract=on" "-fno-rounding-math" "-mconstructor-aliases" "-funwind-tables=2" "-target-cpu" "x86-64" "-mllvm" "-x86-asm-syntax=intel" "-tune-cpu" "generic" "-D_MT" "-D_DLL" "--dependent-lib=msvcrt" "--dependent-lib=oldnames" "--dependent-lib=sycl" "-stack-protector" "2" "-fcxx-exceptions" "-fexceptions" "-fexternc-nounwind" "-fms-volatile" "-fdiagnostics-format" "msvc" "-fcoverage-compilation-dir=D:\\Jakub\\Tickets\\5028" "-resource-dir" "C:\\Users\\oneapi-core\\user_files\\jakub\\llvm\\build\\lib\\clang\\14.0.0" "-internal-isystem" "C:\\Users\\oneapi-core\\user_files\\jakub\\llvm\\build\\bin\\..\\include\\sycl" "-internal-isystem" "C:\\Users\\oneapi-core\\user_files\\jakub\\llvm\\build\\bin\\..\\include" "-iquote" "D:\\Jakub\\Tickets\\5028" "-internal-isystem" "C:\\Users\\oneapi-core\\user_files\\jakub\\llvm\\build\\lib\\clang\\14.0.0\\include" "-internal-isystem" "C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\Community\\VC\\Tools\\MSVC\\14.29.30133\\ATLMFC\\include" "-internal-isystem" "C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\Community\\VC\\Tools\\MSVC\\14.29.30133\\include" "-internal-isystem" "C:\\Program Files (x86)\\Windows Kits\\10\\include\\10.0.19041.0\\ucrt" "-internal-isystem" "C:\\Program Files (x86)\\Windows Kits\\10\\include\\10.0.19041.0\\shared" "-internal-isystem" "C:\\Program Files (x86)\\Windows Kits\\10\\include\\10.0.19041.0\\um" "-internal-isystem" "C:\\Program Files (x86)\\Windows Kits\\10\\include\\10.0.19041.0\\winrt" "-internal-isystem" "C:\\Program Files (x86)\\Windows Kits\\10\\include\\10.0.19041.0\\cppwinrt" "-internal-isystem" "C:\\Users\\oneapi-core\\user_files\\jakub\\llvm\\build\\bin\\..\\include\\sycl" "-internal-isystem" "C:\\Users\\oneapi-core\\user_files\\jakub\\llvm\\build\\bin\\..\\include" "-internal-isystem" "C:\\Users\\oneapi-core\\user_files\\jakub\\llvm\\build\\lib\\clang\\14.0.0\\include" "-internal-isystem" "C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\Community\\VC\\Tools\\MSVC\\14.29.30133\\ATLMFC\\include" "-internal-isystem" "C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\Community\\VC\\Tools\\MSVC\\14.29.30133\\include" "-internal-isystem" "C:\\Program Files (x86)\\Windows Kits\\10\\include\\10.0.19041.0\\ucrt" "-internal-isystem" "C:\\Program Files (x86)\\Windows Kits\\10\\include\\10.0.19041.0\\shared" "-internal-isystem" "C:\\Program Files (x86)\\Windows Kits\\10\\include\\10.0.19041.0\\um" "-internal-isystem" "C:\\Program Files (x86)\\Windows Kits\\10\\include\\10.0.19041.0\\winrt" "-internal-isystem" "C:\\Program Files (x86)\\Windows Kits\\10\\include\\10.0.19041.0\\cppwinrt" "-fdeprecated-macro" "-fdebug-compilation-dir=D:\\Jakub\\Tickets\\5028" "-ferror-limit" "19" "-fno-use-cxa-atexit" "-fms-extensions" "-fms-compatibility" "-fms-compatibility-version=19.29.30136" "-std=c++17" "-fdelayed-template-parsing" "-faddrsig" "-o" "C:\\Users\\ONEAPI~1\\AppData\\Local\\Temp\\main-c3de17.obj" "-x" "c++" "C:\\Users\\ONEAPI~1\\AppData\\Local\\Temp\\main-e63d78.cpp"
 "C:\\Users\\oneapi-core\\user_files\\jakub\\llvm\\build\\bin\\clang-offload-bundler" "-type=o" "-targets=sycl-nvptx64-nvidia-cuda-sm_50,host-x86_64-pc-windows-msvc" "-outputs=main.obj" "-inputs=C:\\Users\\ONEAPI~1\\AppData\\Local\\Temp\\main-cbef33\\main-sm_50.bc,C:\\Users\\ONEAPI~1\\AppData\\Local\\Temp\\main-c3de17.obj"

Candidate path: /Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.4
Candidate path: /Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.3
Candidate path: /Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.2
Candidate path: /Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.1
Candidate path: /Program Files/NVIDIA GPU Computing Toolkit/CUDA/v10.2
Candidate path: /Program Files/NVIDIA GPU Computing Toolkit/CUDA/v10.1
Candidate path: /Program Files/NVIDIA GPU Computing Toolkit/CUDA/v10.0
Candidate path: /Program Files/NVIDIA GPU Computing Toolkit/CUDA/v9.2
Candidate path: /Program Files/NVIDIA GPU Computing Toolkit/CUDA/v9.1
Candidate path: /Program Files/NVIDIA GPU Computing Toolkit/CUDA/v9.0
Candidate path: /Program Files/NVIDIA GPU Computing Toolkit/CUDA/v8.0
Candidate path: /Program Files/NVIDIA GPU Computing Toolkit/CUDA/v7.5
Candidate path: /Program Files/NVIDIA GPU Computing Toolkit/CUDA/v7.0
Candidate path: "------------------>/WOOOF/------------------->"

[bader]

It looks like the right option to use here is -v, not -###. See clang/test/Driver/cuda-detect-path.cu or clang/test/Driver/cuda-detect.cu.
Also, you can use "valid" paths - they are available in clang/test/Driver/Inputs/CUDA/...

[jchlanda]

It looks like the right option to use here is -v, not -###. See clang/test/Driver/cuda-detect-path.cu or clang/test/Driver/cuda-detect.cu. Also, you can use "valid" paths - they are available in clang/test/Driver/Inputs/CUDA/...

That's a great shout, added in: 7e86002

[jchlanda]

@bader I thikn it is needed, as without it the compilation fails (because of the dummy libdevice file):

fatal error: cannot open file 'C:\Users\oneapi-core\user_files\jakub\llvm\clang\test\Driver\Inputs\CUDA_111\usr\local\cuda/nvvm/libdevice/libdevice.10.bc': file too small to contain bitcode header

and as a result FileCheck just propagates the error:

$ "env" "CUDA_PATH=C:\Users\oneapi-core\user_files\jakub\llvm\clang\test\Driver\Inputs\CUDA_111\usr\local\cuda" "c:\users\oneapi-core\user_files\jakub\llvm\build\bin\clang.exe" "-fsycl" "-fsycl-targets=nvptx64-nvidia-cuda" "-v" "C:\Users\oneapi-core\user_files\jakub\llvm\clang\test\Driver\cuda-detect.cu"
note: command had no output on stdout or stderr
error: command failed with exit status: 1
$ "c:\users\oneapi-core\user_files\jakub\llvm\build\bin\filecheck.exe" "C:\Users\oneapi-core\user_files\jakub\llvm\clang\test\Driver\cuda-detect.cu" "-check-prefix" "WINDOWS_CUDA_PATH"

Actually, I was just thinking, this test should stay in a separate file, as it requires system-windows, which is not true for the rest of the discovery tests?

[bader]

Thanks for the clarification. Let's keep this test case in a separate file then.