CUDA compile error

[postacik]

I downloaded the latest master branch and tried to generate build files with CUDA support.

Below you can find the output of the cmake command:

https://pastebin.com/ifjik7wX

Can you please tell me what the problem may be?

I can successfully generate build files without CUDA support.

[takuya-takeuchi]

You must not use latest VS 2017 for CUDA 9.2,
https://devtalk.nvidia.com/default/topic/1035535/cuda-setup-and-installation/cuda-9-2-does-not-work-with-visual-studio-2017-15-7-1/2

Therefore, I use VS 2015 when using CUDA for DlibDotNet.

Nvidia does not already resolve this issue :(
Please use VS 2015 or old VS 2017.

[postacik]

I made the following modification as stated in the page you pointed. I already had VS2015 C++ compiler installed.

C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.2\include\crt\host_config.h
#if _MSC_VER < 1600 /* REMOVE THIS-> || _MSC_VER > 1913 */

DlibDotNet.Native DLL compiled successfully.

However I got the following error for DlibDotNet.Native.Dnn DLL:

ClCompile:
  C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.14.26428\bin\HostX86\x64\CL.exe /c /I"
  C:\Libs\dlib-19.13\dlib\.." /nologo /W3 /WX- /diagnostics:classic /O2 /Ob2 /D "WIN32" /D "_WINDOWS" /D NDEBUG /D "DLI
  B_HAVE_SSE2" /D "DLIB_JPEG_SUPPORT" /D "DLIB_USE_CUDA" /D "DLIB_PNG_SUPPORT" /D "CMAKE_INTDIR=\"Release\"" /D DlibDot
  Net_Native_Dnn_EXPORTS /D "_WINDLL" /D _MBCS /Gm- /EHsc /MD /GS /fp:precise /Zc:wchar_t /Zc:forScope /Zc:inline /GR /
  Fo"DlibDotNet.Native.Dnn.dir\Release\\" /Fd"DlibDotNet.Native.Dnn.dir\Release\vc141.pdb" /Gd /TP /FC /errorReport:que
  ue  /bigobj C:\Users\XXXXXXXX\Desktop\face_recognition\DlibDotNet3\src\DlibDotNet.Native.Dnn\dlib\dnn\loss\loss_metri
  c.cpp C:\Users\XXXXXXXX\Desktop\face_recognition\DlibDotNet3\src\DlibDotNet.Native.Dnn\dlib\dnn\loss\loss_mmod.cpp C:
  \Users\XXXXXXXX\Desktop\face_recognition\DlibDotNet3\src\DlibDotNet.Native.Dnn\dlib\dnn\loss\loss_multiclass_log.cpp
  C:\Users\XXXXXXXX\Desktop\face_recognition\DlibDotNet3\src\DlibDotNet.Native.Dnn\dlib\dnn\loss\loss_multiclass_log_pe
  r_pixel.cpp C:\Users\XXXXXXXX\Desktop\face_recognition\DlibDotNet3\src\DlibDotNet.Native.Dnn\dlib\dnn\output.cpp C:\U
  sers\XXXXXXXX\Desktop\face_recognition\DlibDotNet3\src\DlibDotNet.Native.Dnn\dlib\dnn\trainer.cpp
  loss_metric.cpp
  loss_mmod.cpp
  loss_multiclass_log.cpp
  loss_multiclass_log_per_pixel.cpp
c:\libs\dlib-19.13\dlib\dnn\core.h(919): fatal error C1060: compiler is out of heap space [C:\Users\XXXXXXXX\Desktop\fa
ce_recognition\DlibDotNet3\src\DlibDotNet.Native.Dnn\build\DlibDotNet.Native.Dnn.vcxproj]
  output.cpp
  trainer.cpp
  Generating Code...
Done Building Project "C:\Users\XXXXXXXX\Desktop\face_recognition\DlibDotNet3\src\DlibDotNet.Native.Dnn\build\DlibDotNe
t.Native.Dnn.vcxproj" (default targets) -- FAILED.

Done Building Project "C:\Users\XXXXXXXX\Desktop\face_recognition\DlibDotNet3\src\DlibDotNet.Native.Dnn\build\ALL_BUILD
.vcxproj" (default targets) -- FAILED.


Build FAILED.

"C:\Users\XXXXXXXX\Desktop\face_recognition\DlibDotNet3\src\DlibDotNet.Native.Dnn\build\ALL_BUILD.vcxproj" (default tar
get) (1) ->
"C:\Users\XXXXXXXX\Desktop\face_recognition\DlibDotNet3\src\DlibDotNet.Native.Dnn\build\DlibDotNet.Native.Dnn.vcxproj"
(default target) (3) ->
(ClCompile target) ->
  c:\libs\dlib-19.13\dlib\dnn\core.h(919): fatal error C1060: compiler is out of heap space [C:\Users\XXXXXXXX\Desktop\
face_recognition\DlibDotNet3\src\DlibDotNet.Native.Dnn\build\DlibDotNet.Native.Dnn.vcxproj]

    0 Warning(s)
    1 Error(s)

Do I have to change anything in the cmake command line?

cmake -G "Visual Studio 15 2017 Win64" -DDLIB_PATH="C:\Libs\dlib-19.13" -D CMAKE_PREFIX_PATH="C:\Libs\cuda" ..

IMPORTANT NOTE: I get the same error with the latest master branch when I try to build without CUDA support, too! However I could compile an older version of your library with CUDA support without any problems.

[takuya-takeuchi]

In the first your report, you specified,

cmake -G "Visual Studio 15 2017 Win64" -DDLIB_PATH="C:\dlib-19.13" -D CMAKE_PREFIX_PATH="C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.2" ..

I already wrote the wiki how to compile DlibDotNet.Native.Dnn.
https://github.com/takuya-takeuchi/DlibDotNet/wiki/Tutorial-for-using-CUDA

It indicates the a certain flag nnceremoniously.

mkdir build
cd build
cmake -G "Visual Studio 15 2017 Win64" -T host=x64 ^
      -D DLIB_PATH=D:/Works/Lib/DLib/19.8 ^
      -D CMAKE_PREFIX_PATH=D:/Works/Lib/NVIDIA/cuDNN/9.1/7.0/Win10 ^
      ..

-T host=x64 flags commands to compiler for using x64 tool chain.
It does NOT mean that generating x64 binary.
x64 tool chain can reserve large heap memory and it will be all right!!!

[postacik]

Sorry, I've missed that flag in the cmake command.

Now I've compiled both native libraries with CUDA support using the latest master branch.

I looked at the dependencies of the dlls and found 2 CUDA libraries:
cublas64_92.dll
cudnn64_7.dll

I copied the native libraries and the 2 CUDA libraries to the binary folder of the DnnFaceRecognition sample but I get an error at the following line:

            // And finally we load the DNN responsible for face recognition.
            using (var net = DlibDotNet.Dnn.LossMetric.Deserialize("dlib_face_recognition_resnet_model_v1.dat"))

System.Runtime.InteropServices.SEHException: 'External component has thrown an exception.'

Are there any missing CUDA dependencies? What am I missing?

[takuya-takeuchi]

It is weird. I just tested DnnFaceRecognition for cuda 9.1 and it has no error.
I want to believe cuda version does not matter....

LossMetric.Deserialize is simple wrap method.
Could you try other DnnXXXXX sample project?
I think DnnMmodFaceDetection is 2nd simple project.
Needless to say, I have already confirmed it is fine!!

If you will face same issue, I may have to install CUDA 9.2 and test it.

[postacik]

I tested the DnnMmodFaceDetection sample.

It gives an exception at the same place:

var ret = Native.loss_mmod_deserialize(str, networkType);

This is the contents of the bin directory of the project:

05.06.2018  18:07           648.373 bald_guys.jpg
07.05.2018  09:14        54.330.880 cublas64_92.dll
15.07.2018  14:35       336.443.392 cudnn64_7.dll
15.07.2018  22:18           256.512 DlibDotNet.dll
15.07.2018  22:22         7.820.288 DlibDotNet.Native.dll
15.07.2018  22:14         5.008.384 DlibDotNet.Native.Dnn.dll
15.07.2018  22:18           116.324 DlibDotNet.pdb
15.07.2018  22:18               778 DnnMmodFaceDetection.deps.json
15.07.2018  22:18             6.656 DnnMmodFaceDetection.dll
15.07.2018  22:18             1.024 DnnMmodFaceDetection.pdb
15.07.2018  22:18               242 DnnMmodFaceDetection.runtimeconfig.dev.json
15.07.2018  22:18               154 DnnMmodFaceDetection.runtimeconfig.json
16.07.2018  09:28           729.940 mmod_human_face_detector.dat

I'm using dlib 19.13 and CUDA 9.2.

When I compile DlibDotNet.Native.Dnn.dll without CUDA support its size is 1.052 KB.
When I compile DlibDotNet.Native.Dnn.dll with CUDA support its size is 4.891 KB.

When I compile DlibDotNet.Native.dll without CUDA support its size is 6.006 KB.
When I compile DlibDotNet.Native.dll with CUDA support its size is 7.637 KB.

Is this normal?

Note: I'm working on a Dell Precision laptop and it has 2 display adapters:
Intel HD Graphics 630
NVIDIA Quadro M620

[postacik]

I compiled dlib examples with CUDA support and tried to run dnn_face_recognition_ex.exe application.

I got the following error:

C:\Libs\dlib-19.15\examples\build\Release>dnn_face_recognition_ex.exe bald_guys.jpg
Error while calling cudaGetDevice(&the_device_id) in file c:\libs\dlib-19.15\dlib\cuda\gpu_data.cpp:178. code: 35, reason: CUDA driver version is insufficient for CUDA runtime version

Can this be the reason for the error in your library?

[postacik]

Ok, I finally solved it.

While installing CUDA, I had unchecked the graphics driver installation option.

I reinstalled CUDA and installed the driver this time.

Now everything works fine and FAST with the latest master branch.

Thanks for your time and support. 👍

Note: I still think there's a problem with the HUGE FILE SIZE of the DlibDotNet.Native.Dnn.dll file with CUDA support. It may be a good idea to check it.