Build ceres_cuda_kernels failed: nvcc fatal #887

WuKongBlog · 2022-09-20T02:57:34Z

Build latest ceres with CUDA on windows 10 failed

Log：

D:\Develop\library\ceres-solver\BUILD-2\internal\ceres>"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.6\bin\nvcc.exe" -gencode=arch=compute_50,code=\"compute_50,compute_50\" -gencode=arch=compute_50,code=\"sm_50,compute_50\" -gencode=arch=compute_60,code=\"compute_60,compute_60\" -gencode=arch=compute_60,code=\"sm_60,compute_60\" -gencode=arch=compute_70,code=\"compute_70,compute_70\" -gencode=arch=compute_70,code=\"sm_70,compute_70\" -gencode=arch=compute_80,code=\"compute_80,compute_80\" -gencode=arch=compute_80,code=\"sm_80,compute_80\" --use-local-env -ccbin "D:\Programs\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\bin\HostX86\x64" -x cu    -I"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.6\include"     --keep-dir x64\Release  -maxrregcount=0  --machine 64 --compile -cudart static /wd4018 /wd4267 /wd4099 /wd4996 /wd4800 /wd4244 /wd4251 /bigobj -std=c++14 -Xcompiler="/EHsc -Ob2"   -D_WINDOWS -DNDEBUG -D_USE_MATH_DEFINES -D_VARIADIC_MAX=10 -DNOMINMAX -D"CMAKE_INTDIR=\"Release\"" -D_MBCS -D"CMAKE_INTDIR=\"Release\"" -Xcompiler "/EHsc /W3 /nologo /O2 /Fdceres_cuda_kernels.dir\Release\ceres_cuda_kernels.pdb /FS   /MD /GR" -o ceres_cuda_kernels.dir\Release\cuda_kernels.cu.obj "D:\Develop\library\ceres-solver\internal\ceres\cuda_kernels.cu.cc"

nvcc fatal   : A single input file is required for a non-link phase when an outputfile is specified

I did some searches and found that this error may be caused by the space in the path, but there is no space in my path.
Is it because the path after /Fd does not have quotation marks?

PS: I use visual studio 2017

The text was updated successfully, but these errors were encountered:

sandwichmaker · 2022-09-20T03:16:36Z

Cc:@joydeep-b

joydeep-b · 2022-09-20T13:01:40Z

@WuKongBlog What happens when you try to execute that line manually from the terminal, while in the build directory?

joydeep-b · 2022-09-20T15:29:32Z

Another thought (based on the "a single input file" part of the error) : I wonder if the ".cu.cc" extension is throwing it off. Could you also try renaming it to just ".cu" and re-run the command manually with just the ".cu" extension?

WuKongBlog · 2022-09-21T01:00:07Z

I tried to execute the original command, the command after add quotation marks after/Fd, and the command after remove ". cc" in the build path.
The same error was reported:（

joydeep-b · 2022-09-21T01:07:40Z

Hmm, this is mystifying, but I'm also not sure about all the VS flags. Could you try:

"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.6\bin\nvcc.exe"  -o ceres_cuda_kernels.dir\Release\cuda_kernels.cu.obj "D:\Develop\library\ceres-solver\internal\ceres\cuda_kernels.cu.cc"

WuKongBlog · 2022-09-21T01:10:56Z

:(

joydeep-b · 2022-09-21T01:36:09Z

Ouch.
Okay, on linux, the following succeeds:

nvcc --compile -std=c++14 -x cu -o internal/ceres/CMakeFiles/ceres_cuda_kernels.dir/cuda_kernels.cu.cc.o ../internal/ceres/cuda_kernels.cu.cc

I think the following would be the Windows equivalent, can you try that?

"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.6\bin\nvcc.exe"  --compile -std=c++14 -x cu -o ceres_cuda_kernels.dir\Release\cuda_kernels.cu.obj "D:\Develop\library\ceres-solver\internal\ceres\cuda_kernels.cu.cc"

WuKongBlog · 2022-09-21T02:17:58Z

After my line by line analysis, I found the problem. The command on the right can run successfully.
For nvcc fatal: A single input file is required for a non link phase when an outputfile is specified : /wd4018/wd4267/wd4099/wd4996/wd4800/wd4244/wd4251/bigobj should not appear behind the -cudart static, though I don't know where to put this...
There are also some other problems, see the figure for details:

So, how should I modify CMakeLists according to this :)

PS: The New Command

"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.6\bin\nvcc.exe" -gencode=arch=compute_50,code=\"compute_50,compute_50\" -gencode=arch=compute_50,code=\"sm_50,compute_50\" -gencode=arch=compute_60,code=\"compute_60,compute_60\" -gencode=arch=compute_60,code=\"sm_60,compute_60\" -gencode=arch=compute_70,code=\"compute_70,compute_70\" -gencode=arch=compute_70,code=\"sm_70,compute_70\" -gencode=arch=compute_80,code=\"compute_80,compute_80\" -gencode=arch=compute_80,code=\"sm_80,compute_80\" -ccbin "D:\Programs\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\bin\HostX86\x64" -x cu -I"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.6\include" --keep-dir x64\Release -maxrregcount=0 --machine 64 --compile -cudart static -std=c++17 -Xcompiler="/EHsc -Ob2" -D_WINDOWS -DNDEBUG -D_USE_MATH_DEFINES -D_VARIADIC_MAX=10 -DNOMINMAX -D"CMAKE_INTDIR=\"Release\"" -D_MBCS -D"CMAKE_INTDIR=\"Release\"" -Xcompiler "/EHsc /W3 /nologo /O2 /Fdceres_cuda_kernels.dir\Release\ceres_cuda_kernels.pdb /FS   /MD /GR" -o ceres_cuda_kernels.dir\Release\cuda_kernels.cu.obj "D:\Develop\library\ceres-solver\internal\ceres\cuda_kernels.cu.cc"

joydeep-b · 2022-09-22T17:22:02Z

@WuKongBlog Thanks, this is very helpful!
Can you share the compiler error that you get when you compile with "--std=c++14"?

Also, what happens if you include all the "/wd*" flags but omit the "/bigobj" flag?

prasadvagdargi · 2022-09-27T01:00:49Z

Temporarily fixed the issue by removing /bigobj and the /wd* flags and compiling with --std=c++17.

The error with 1) all /wd* flags and --std=c++14 2) all /wd* flags omitting /bigobj 3) All /wd* flags with --std=c++17 4) /bigobj flag and -std=c++17
remains:

MSB3721 The command ""C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.3\bin\nvcc.exe" -gencode=arch=compute_50,code="compute_50,compute_50" -gencode=arch=compute_50,code="sm_50,compute_50" -gencode=arch=compute_60,code="compute_60,compute_60" -gencode=arch=compute_60,code="sm_60,compute_60" -gencode=arch=compute_70,code="compute_70,compute_70" -gencode=arch=compute_70,code="sm_70,compute_70" -gencode=arch=compute_80,code="compute_80,compute_80" -gencode=arch=compute_80,code="sm_80,compute_80" --use-local-env -ccbin "C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\bin\HostX86\x64" -x cu -I"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.3\include" --keep-dir x64\Release -maxrregcount=0 --machine 64 --compile -cudart static /wd4018 /wd4267/ wd4099/ wd4996/ wd4800/ wd4244/ wd4251 /bigobj-std=c++14 -Xcompiler="/EHsc -Ob2" -D_WINDOWS -DNDEBUG -D_USE_MATH_DEFINES -D_VARIADIC_MAX=10 -DNOMINMAX -D"CERES_METIS_VERSION="5.1.0"" -D"CMAKE_INTDIR="Release"" -D"CMAKE_INTDIR="Release"" -D_MBCS -Xcompiler "/EHsc /W3 /nologo /O2 /Fdceres_cuda_kernels.dir\Release\ceres_cuda_kernels.pdb /FS /MD /GR" -o ceres_cuda_kernels.dir\Release\cuda_kernels.cu.obj "C:\istar\extras\ceres-solver\internal\ceres\cuda_kernels.cu.cc"" exited with code 1. ceres_cuda_kernels C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\Common7\IDE\VC\VCTargets\BuildCustomizations\CUDA 11.3.targets Line: 785

Keeping the /bigobj flag and --std==c+14 results in:

..ceres-solver\internal\ceres\cuda_kernels.cu.cc(33): error : qualified name is not allowed
C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\Common7\IDE\VC\VCTargets\BuildCustomizations\CUDA 11.3.targets(785,9): error MSB3721: The command ""C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.3\bin\nvcc.exe" -gencode=arch=compute_50,code="compute_50,compute_50" -gencode=arch=compute_50,code="sm_50,compute_50" -gencode=arch=compute_60,code="compute_60,compute_60" -gencode=arch=compute_60,code="sm_60,compute_60" -gencode=arch=compute_70,code="compute_70,compute_70" -gencode=arch=compute_70,code="sm_70,compute_70" -gencode=arch=compute_80,code="compute_80,compute_80" -gencode=arch=compute_80,code="sm_80,compute_80" --use-local-env -ccbin "C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\bin\HostX86\x64" -x cu -I"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.3\include" --keep-dir x64\Release -maxrregcount=0 --machine 64 --compile -cudart static -std=c++14 -Xcompiler="/EHsc -Ob2" -D_WINDOWS -DNDEBUG -D_USE_MATH_DEFINES -D_VARIADIC_MAX=10 -DNOMINMAX -D"CERES_METIS_VERSION="5.1.0"" -D"CMAKE_INTDIR="Release"" -D"CMAKE_INTDIR="Release"" -D_MBCS -Xcompiler "/EHsc /W3 /nologo /O2 /Fdceres_cuda_kernels.dir\Release\ceres_cuda_kernels.pdb /FS /MD /GR" -o ceres_cuda_kernels.dir\Release\cuda_kernels.cu.obj "C:\istar\extras\ceres-solver\internal\ceres\cuda_kernels.cu.cc"" exited with code 1.

WuKongBlog · 2022-09-27T01:22:57Z

@joydeep-b

Compile with "--std=c++14"

Remove the -x cu , more compilation error information can be obtained like this:

D:\Develop\library\ceres-solver\build-wsh\internal\ceres>"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.6\bin\nvcc.exe" -gencode=arch=compute_50,code=\"compute_50,compute_50\" -gencode=arch=compute_50,code=\"sm_50,compute_50\" -gencode=arch=compute_60,code=\"compute_60,compute_60\" -gencode=arch=compute_60,code=\"sm_60,compute_60\" -gencode=arch=compute_70,code=\"compute_70,compute_70\" -gencode=arch=compute_70,code=\"sm_70,compute_70\" -gencode=arch=compute_80,code=\"compute_80,compute_80\" -gencode=arch=compute_80,code=\"sm_80,compute_80\" -ccbin "D:\Programs\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\bin\HostX86\x64" -I"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.6\include" --keep-dir x64\Release -maxrregcount=0 --machine 64 --compile -cudart static -std=c++14 -Xcompiler="/EHsc -Ob2" -D_WINDOWS -DNDEBUG -D_USE_MATH_DEFINES -D_VARIADIC_MAX=10 -DNOMINMAX -D"CMAKE_INTDIR=\"Release\"" -D_MBCS -D"CMAKE_INTDIR=\"Release\"" -Xcompiler "/EHsc /W3 /nologo /O2 /Fd"ceres_cuda_kernels.dir\Release\ceres_cuda_kernels.pdb" /FS   /MD /GR" -o ceres_cuda_kernels.dir\Release\cuda_kernels.cu.obj "D:\Develop\library\ceres-solver\internal\ceres\cuda_kernels.cu.cc"

cuda_kernels.cu.cc

D:\Develop\library\ceres-solver\internal\ceres\cuda_kernels.cu.cc(33): error C2429: 语言功能 "嵌套命名空间定义" 需要编译器标志 "/std:c++17"
D:\Develop\library\ceres-solver\internal\ceres\cuda_kernels.cu.cc(59): error C2059: 语法错误:“<”
D:\Develop\library\ceres-solver\internal\ceres\cuda_kernels.cu.cc(68): error C2059: 语法错误:“<”
D:\Develop\library\ceres-solver\internal\ceres\cuda_kernels.cu.cc(81): error C2059: 语法错误:“<”
D:\Develop\library\ceres-solver\internal\ceres\cuda_kernels.cu.cc(87): error C2059: 语法错误:“<”
D:\Develop\library\ceres-solver\internal\ceres\cuda_kernels.cu.cc(103): error C2059: 语法错误:“<”
D:\Develop\library\ceres-solver\internal\ceres\cuda_kernels.cu.cc(110): error C2065: “blockIdx”: 未声明的标识符
D:\Develop\library\ceres-solver\internal\ceres\cuda_kernels.cu.cc(110): error C2065: “blockDim”: 未声明的标识符
D:\Develop\library\ceres-solver\internal\ceres\cuda_kernels.cu.cc(110): error C2065: “threadIdx”: 未声明的标识符
D:\Develop\library\ceres-solver\internal\ceres\cuda_kernels.cu.cc(110): error C2789: “i”: 必须初始化常量限定类型的对象D:\Develop\library\ceres-solver\internal\ceres\cuda_kernels.cu.cc(110): note: 参见“i”的声明
D:\Develop\library\ceres-solver\internal\ceres\cuda_kernels.cu.cc(122): error C2059: 语法错误:“<”

The first error line is about Nested Namespace Definition, which requires c++17

Add "/wd*" "/bigobj" flags

If I add all the "/wd*" flags and the "/bigobj" flag inside the -Xcompiler='...' , the command can run successfully, but I don't know if it's right

The new command:

"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.6\bin\nvcc.exe" ^
-gencode=arch=compute_50,code=\"compute_50,compute_50\" -gencode=arch=compute_50,code=\"sm_50,compute_50\" ^
-gencode=arch=compute_60,code=\"compute_60,compute_60\" -gencode=arch=compute_60,code=\"sm_60,compute_60\" ^
-gencode=arch=compute_70,code=\"compute_70,compute_70\" -gencode=arch=compute_70,code=\"sm_70,compute_70\" ^
-gencode=arch=compute_80,code=\"compute_80,compute_80\" -gencode=arch=compute_80,code=\"sm_80,compute_80\" ^
-ccbin "D:\Programs\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\bin\HostX86\x64" ^
-x cu ^
-I"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.6\include" ^
--keep-dir x64\Release ^
-maxrregcount=0 ^
--machine 64 ^
--compile ^
-cudart static ^
-std=c++17 ^
-Xcompiler="/EHsc /wd4018 /wd4267 /wd4099 /wd4996 /wd4800 /wd4244 /wd4251 /bigobj -Ob2" ^
-D_WINDOWS ^
-DNDEBUG ^
-D_USE_MATH_DEFINES ^
-D_VARIADIC_MAX=10 ^
-DNOMINMAX ^
-D"CMAKE_INTDIR=\"Release\"" ^
-D_MBCS ^
-D"CMAKE_INTDIR=\"Release\"" ^
-Xcompiler "/EHsc /W3 /nologo /O2 /Fd"ceres_cuda_kernels.dir\Release\ceres_cuda_kernels.pdb" /FS   /MD /GR" ^
-o ceres_cuda_kernels.dir\Release\cuda_kernels.cu.obj ^
"D:\Develop\library\ceres-solver\internal\ceres\cuda_kernels.cu.cc"

joydeep-b · 2022-09-27T18:17:41Z

We cannot remove the -x cu flag - it's telling the compiler that this is a C++ file with CUDA extensions. Without it, it will try to compile as just C++ and fail on the kernel invocations and the kernel threading variables (the errors that you listed).
link

The /wd* flags are disabling specific warnings. They should not be the cause of the problem. link

The /bigobj just tells the compiler to support more addressable segments link

@WuKongBlog Did the last command that you pasted work? That seems to indicate that we only need to update the C++ spec to 17, and remove "--use-local-env". Is that correct?

WuKongBlog · 2022-09-29T08:19:56Z

@joydeep-b

Of course we can't remove the -x cu flag，I removed this command just to show more compilation errors, like the error about the c++version.
There is no problem with the /wd* and /bigobj flags themselves. The problem is their location. If they are placed behind the -cudart static like before, you will be prompted that nvcc fatal: A single input file is required for a non link phase when an outputfile is specified.
Yes, the last command works. Actually, there are three changes:
- update the C++ spec to 17
- remove --use-local-env
- move the /wd* and /bigobj flags to -Xcompiler="..."
  I don't know whether it is correct to put the /wd* and /bigobj flags in -Xcompiler="...", but they can't be placed behind -cudart static directly

sandwichmaker · 2022-11-25T22:31:30Z

@joydeep-b whats the resolution here?

joydeep-b · 2022-11-26T14:25:34Z

CC'ing @sergiud since he has a Windows box with CUDA, and CMake experience.

The /wd* flags are coming from here: https://github.com/ceres-solver/ceres-solver/blob/master/CMakeLists.txt#L545
One option is to completely strip the compile options for the nvcc compiler, and pass in a different set of options. Not sure what the best way to do this is with CMake.
The other option is to somehow tell CMake the ordering of these options when passing them to nvcc. Again, not sure how this is done.

@sergiud What do you think?

sandwichmaker · 2023-09-21T19:36:22Z

Is this still an issue? I am going to close it because of lack of updates and assume that 2.2.0 solves this, if not please reopen.

sandwichmaker assigned joydeep-b Sep 22, 2022

Adela0814 mentioned this issue Oct 13, 2022

[ceres] Remove cxsparse and add metis support microsoft/vcpkg#27210

Closed

sandwichmaker added the NVIDIA CUDA Nvidia GPU related issues label Nov 25, 2022

sandwichmaker closed this as completed Sep 21, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Build ceres_cuda_kernels failed: nvcc fatal #887

Build ceres_cuda_kernels failed: nvcc fatal #887

WuKongBlog commented Sep 20, 2022 •

edited

sandwichmaker commented Sep 20, 2022

joydeep-b commented Sep 20, 2022

joydeep-b commented Sep 20, 2022

WuKongBlog commented Sep 21, 2022

joydeep-b commented Sep 21, 2022

WuKongBlog commented Sep 21, 2022

joydeep-b commented Sep 21, 2022

WuKongBlog commented Sep 21, 2022 •

edited

joydeep-b commented Sep 22, 2022

prasadvagdargi commented Sep 27, 2022

WuKongBlog commented Sep 27, 2022 •

edited

joydeep-b commented Sep 27, 2022

WuKongBlog commented Sep 29, 2022

sandwichmaker commented Nov 25, 2022

joydeep-b commented Nov 26, 2022

sandwichmaker commented Sep 21, 2023

Build ceres_cuda_kernels failed: nvcc fatal #887

Build ceres_cuda_kernels failed: nvcc fatal #887

Comments

WuKongBlog commented Sep 20, 2022 • edited

sandwichmaker commented Sep 20, 2022

joydeep-b commented Sep 20, 2022

joydeep-b commented Sep 20, 2022

WuKongBlog commented Sep 21, 2022

joydeep-b commented Sep 21, 2022

WuKongBlog commented Sep 21, 2022

joydeep-b commented Sep 21, 2022

WuKongBlog commented Sep 21, 2022 • edited

joydeep-b commented Sep 22, 2022

prasadvagdargi commented Sep 27, 2022

WuKongBlog commented Sep 27, 2022 • edited

joydeep-b commented Sep 27, 2022

WuKongBlog commented Sep 29, 2022

sandwichmaker commented Nov 25, 2022

joydeep-b commented Nov 26, 2022

sandwichmaker commented Sep 21, 2023

WuKongBlog commented Sep 20, 2022 •

edited

WuKongBlog commented Sep 21, 2022 •

edited

WuKongBlog commented Sep 27, 2022 •

edited