Problem compiling mexGPUall

[Kasia-Bz]

Hello,
I am trying to install Kilosort2 on my new laptop but I am running into problems. I installed Matlab 2020a and CUDA11 (it was the only version which allowed for CUDA to recognize my GPU). I used both Visual Studio 2015 and 2019, but always getting the same problem.
When I run mexGPUall I get the following output:

``Building with 'NVIDIA CUDA Compiler'.
MEX completed successfully.
Building with 'NVIDIA CUDA Compiler'.
MEX completed successfully.
Building with 'NVIDIA CUDA Compiler'.
Error using mex
fatbinary fatal   : '19_device.compute_35.cubin' is not in 'keyword=value' format
mexMPnu8.obj
c_mexapi_version.obj
Error in mexcuda (line 166)
    [varargout{1:nargout}] = mex(mexArguments{:});

Error in mexGPUall (line 16)
        mexcuda -largeArrayDims -dynamic -DENABLE_STABLEMODE mexMPnu8.cu

When I run: gpuDevice(1) I get following:

CUDADevice with properties:

                  Name: 'GeForce GTX 1660 Ti'
                 Index: 1
     ComputeCapability: '7.5'
        SupportsDouble: 1
         DriverVersion: 11
        ToolkitVersion: 10.1000
    MaxThreadsPerBlock: 1024
      MaxShmemPerBlock: 49152
    MaxThreadBlockSize: [1024 1024 64]
           MaxGridSize: [2.1475e+09 65535 65535]
             SIMDWidth: 32
           TotalMemory: 6.4425e+09
       AvailableMemory: 5.1299e+09
   MultiprocessorCount: 24
          ClockRateKHz: 1590000
           ComputeMode: 'Default'
  GPUOverlapsTransfers: 1
KernelExecutionTimeout: 1
      CanMapHostMemory: 1
       DeviceSupported: 1
        DeviceSelected: 1

When I run those commands:
gpuEnvObj = coder.gpuEnvConfig;

gpuEnvObj.BasicCodegen = 1;
gpuEnvObj.BasicCodeexec = 1;
results = coder.checkGpuInstall(gpuEnvObj)
Compatible GPU : PASSED
CUDA Environment : PASSED
Runtime : PASSED
cuFFT : PASSED
cuSOLVER : PASSED
cuBLAS : PASSED

I guess that would mean that CUDA and GPU are compatible, yet I cannot compile the mex file.
I also previously installed Kilosort2 on different pc, with Matlab 2019b and CUDA10.1 with Visual Studio 2015, I tried the same configuration on my current laptop, but in that case CUDA was failing to recognized my GPU and I was able to fix it only by installing CUDA11.

Does someone have an idea how to solve this issue?

Thank you in advance!

[marius10p]

Looks like it's compiling some of the mex files but failing on the third one. For now, go into mexGPUall and set enableStableMode = 0.

@jenniferColonell any idea why this is not compiling?

[jenniferColonell]

I am not sure what the issue is. If you try the compile with enableStableMode = 1 and verbose turned on, MATLAB will display more details and we might be able to diagnose the problem.

You'll need to edit the line in mexGPUall.m:

mexcuda -v -largeArrayDims -dynamic -DENABLE_STABLEMODE mexMPnu8.cu

[Kasia-Bz]

Here is the output when I compile with verbose on:

Building with 'NVIDIA CUDA Compiler'.
MEX completed successfully.
Building with 'NVIDIA CUDA Compiler'.
MEX completed successfully.
Trying MEX options 'C:\Program Files\MATLAB\R2020a\toolbox\parallel\gpu\extern\src\mex\win64\nvcc_msvcpp2015_dynamic.xml'...SUCCESS
mex -largeArrayDims -f C:\Program Files\MATLAB\R2020a\toolbox\parallel\gpu\extern\src\mex\win64\nvcc_msvcpp2015_dynamic.xml NVCC_FLAGS="" -v -largeArrayDims -DENABLE_STABLEMODE mexMPnu8.cu 
Verbose mode is on.
... Looking for compiler 'NVIDIA CUDA Compiler' ...
... Looking for registry setting 'HKLM\SOFTWARE\Microsoft\VisualStudio\SxS\VC7' 14.0 ...No.
... Looking for registry setting 'HKCU\SOFTWARE\Microsoft\VisualStudio\SxS\VC7' 14.0 ...No.
... Looking for registry setting 'HKLM\SOFTWARE\Wow6432Node\Microsoft\VisualStudio\SxS\VC7' 14.0 ...Yes ('C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\').
... Looking for file 'C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\bin\amd64\cl.exe' ...Yes.
... Looking for folder 'C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC' ...Yes.
... Looking for registry setting 'HKLM\SOFTWARE\Microsoft\Microsoft SDKs\Windows\v8.1' InstallationFolder ...No.
... Looking for registry setting 'HKCU\SOFTWARE\Microsoft\Microsoft SDKs\Windows\v8.1' InstallationFolder ...No.
... Looking for registry setting 'HKLM\SOFTWARE\Wow6432Node\Microsoft\Microsoft SDKs\Windows\v8.1' InstallationFolder ...Yes ('C:\Program Files (x86)\Windows Kits\8.1\').
... Looking for registry setting 'HKLM\SOFTWARE\Microsoft\VisualStudio\SxS\VS7' 14.0 ...No.
... Looking for registry setting 'HKCU\SOFTWARE\Microsoft\VisualStudio\SxS\VS7' 14.0 ...No.
... Looking for registry setting 'HKLM\SOFTWARE\Wow6432Node\Microsoft\VisualStudio\SxS\VS7' 14.0 ...Yes ('C:\Program Files (x86)\Microsoft Visual Studio 14.0\').
... Looking for registry setting 'HKLM\SOFTWARE\Microsoft\VisualStudio\SxS\VC7' 14.0 ...No.
... Looking for registry setting 'HKCU\SOFTWARE\Microsoft\VisualStudio\SxS\VC7' 14.0 ...No.
... Looking for registry setting 'HKLM\SOFTWARE\Wow6432Node\Microsoft\VisualStudio\SxS\VC7' 14.0 ...Yes ('C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\').
... Looking for registry setting 'HKLM\SOFTWARE\Microsoft\VisualStudio\SxS\VC7' 14.0 ...No.
... Looking for registry setting 'HKCU\SOFTWARE\Microsoft\VisualStudio\SxS\VC7' 14.0 ...No.
... Looking for registry setting 'HKLM\SOFTWARE\Wow6432Node\Microsoft\VisualStudio\SxS\VC7' 14.0 ...Yes ('C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\').
... Looking for file 'C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\vcvarsall.bat' ...Yes.
... Looking for folder 'C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC' ...Yes.
... Looking for registry setting 'HKLM\SOFTWARE\Wow6432Node\Microsoft\Windows Kits\Installed Roots' KitsRoot10 ...Yes ('C:\Program Files (x86)\Windows Kits\10\').
... Looking for registry setting 'HKLM\SOFTWARE\Wow6432Node\Microsoft\Windows Kits\Installed Roots' KitsRoot10 ...Yes ('C:\Program Files (x86)\Windows Kits\10\').
... Executing command 'echo off&set "sdkversion="&(for /f %a IN ('dir "C:\Program Files (x86)\Windows Kits\10\include\" /b /ad-h /on') do ( @if exist "C:\Program Files (x86)\Windows Kits\10\include\%a\ucrt\" set "sdkversion=%a" ))&call echo %sdkversion%' ...Yes ('10.0.18362.0').
... Looking for environment variable 'MW_NVCC_PATH' ...No.
... Looking for environment variable 'CUDA_LIB_PATH' ...No.
... Looking for environment variable 'CUDA_PATH' ...Yes ('C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.0').
... Looking for file 'C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.0\lib\x64\cudart.lib' ...Yes.
... Looking for folder 'C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.0\lib\x64' ...Yes.
... Executing command '(IF DEFINED MW_ALLOW_ANY_CUDA ( true )  ELSE ( false )) > NUL && echo C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.0\lib\x64' ...No.
... Executing command '$$\bin\nvcc --version | findstr /c:"release 10.1" > NUL && echo $$' ...No.
... Looking for environment variable 'MW_NVCC_PATH' ...No.
... Looking for environment variable 'CUDA_LIB_PATH' ...No.
... Looking for environment variable 'CUDA_PATH' ...Yes ('C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.0').
... Looking for file 'C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.0\lib\x64\cudart.lib' ...Yes.
... Looking for folder 'C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.0\lib\x64' ...Yes.
... Executing command '(IF DEFINED MW_ALLOW_ANY_CUDA ( true )  ELSE ( false )) > NUL && echo C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.0\lib\x64' ...No.
... Executing command '"$$\bin\nvcc" --version | findstr /c:"release 10.1" > NUL && echo $$' ...No.
... Looking for file 'C:\Program Files\MATLAB\R2020a\sys\cuda\win64\cuda\lib\x64\cudart.lib' ...Yes.
... Looking for folder 'C:\Program Files\MATLAB\R2020a\sys\cuda\win64\cuda\lib\x64' ...Yes.
... Looking for environment variable 'MW_NVCC_PATH' ...No.
... Looking for environment variable 'CUDA_BIN_PATH' ...No.
... Looking for environment variable 'CUDA_PATH' ...Yes ('C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.0').
... Looking for file 'C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.0\bin\nvcc.exe' ...Yes.
... Looking for folder 'C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.0' ...Yes.
... Executing command '(IF DEFINED MW_ALLOW_ANY_CUDA ( true )  ELSE ( false )) > NUL && echo C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.0' ...No.
... Executing command '$$\bin\nvcc --version | findstr /c:"release 10.1" > NUL && echo $$' ...No.
... Looking for environment variable 'MW_NVCC_PATH' ...No.
... Looking for environment variable 'CUDA_BIN_PATH' ...No.
... Looking for environment variable 'CUDA_PATH' ...Yes ('C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.0').
... Looking for file 'C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.0\bin\nvcc.exe' ...Yes.
... Looking for folder 'C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.0' ...Yes.
... Executing command '(IF DEFINED MW_ALLOW_ANY_CUDA ( true )  ELSE ( false )) > NUL && echo C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.0' ...No.
... Executing command '"$$\bin\nvcc" --version | findstr /c:"release 10.1" > NUL && echo $$' ...No.
... Looking for file 'C:\Program Files\MATLAB\R2020a\sys\cuda\win64\cuda\bin\nvcc.exe' ...Yes.
... Looking for folder 'C:\Program Files\MATLAB\R2020a\sys\cuda\win64\cuda' ...Yes.
... Looking for folder 'C:\Program Files\MATLAB\R2020a\sys\cuda\win64\cuda' ...Yes.
... Looking for environment variable 'MW_NVCC_PATH' ...No.
... Looking for environment variable 'CUDA_BIN_PATH' ...No.
... Looking for environment variable 'CUDA_PATH' ...Yes ('C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.0').
... Looking for file 'C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.0\bin\nvcc.exe' ...Yes.
... Looking for folder 'C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.0\nvvm\libdevice' ...Yes.
... Executing command '(IF DEFINED MW_ALLOW_ANY_CUDA ( true )  ELSE ( false )) > NUL && echo C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.0\nvvm\libdevice' ...No.
... Executing command '$$\bin\nvcc --version | findstr /c:"release 10.1" > NUL && echo $$' ...No.
... Looking for environment variable 'MW_NVCC_PATH' ...No.
... Looking for environment variable 'CUDA_BIN_PATH' ...No.
... Looking for environment variable 'CUDA_PATH' ...Yes ('C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.0').
... Looking for file 'C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.0\bin\nvcc.exe' ...Yes.
... Looking for folder 'C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.0\nvvm\libdevice' ...Yes.
... Executing command '(IF DEFINED MW_ALLOW_ANY_CUDA ( true )  ELSE ( false )) > NUL && echo C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.0\nvvm\libdevice' ...No.
... Executing command '"$$\bin\nvcc" --version | findstr /c:"release 10.1" > NUL && echo $$' ...No.
... Looking for file 'C:\Program Files\MATLAB\R2020a\sys\cuda\win64\cuda\bin\nvcc.exe' ...Yes.
... Looking for folder 'C:\Program Files\MATLAB\R2020a\bin\win64' ...Yes.
... Looking for folder 'C:\Program Files\MATLAB\R2020a\bin\win64' ...Yes.
... Looking for environment variable 'MW_NVCC_PATH' ...No.
... Looking for environment variable 'CUDA_BIN_PATH' ...No.
... Looking for environment variable 'CUDA_PATH' ...Yes ('C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.0').
... Looking for file 'C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.0\bin\nvcc.exe' ...Yes.
... Looking for folder 'C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.0\lib\x64' ...Yes.
... Executing command '(IF DEFINED MW_ALLOW_ANY_CUDA ( true )  ELSE ( false )) > NUL && echo C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.0\lib\x64' ...No.
... Executing command '$$\bin\nvcc --version | findstr /c:"release 10.1" > NUL && echo $$' ...No.
... Looking for environment variable 'MW_NVCC_PATH' ...No.
... Looking for environment variable 'CUDA_BIN_PATH' ...No.
... Looking for environment variable 'CUDA_PATH' ...Yes ('C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.0').
... Looking for file 'C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.0\bin\nvcc.exe' ...Yes.
... Looking for folder 'C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.0\lib\x64' ...Yes.
... Executing command '(IF DEFINED MW_ALLOW_ANY_CUDA ( true )  ELSE ( false )) > NUL && echo C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.0\lib\x64' ...No.
... Executing command '"$$\bin\nvcc" --version | findstr /c:"release 10.1" > NUL && echo $$' ...No.
... Looking for file 'C:\Program Files\MATLAB\R2020a\sys\cuda\win64\cuda\bin\nvcc.exe' ...Yes.
... Looking for folder 'C:\Program Files\MATLAB\R2020a\bin\win64' ...Yes.
... Looking for folder 'C:\Program Files\MATLAB\R2020a\bin\win64' ...Yes.
... Executing command 'echo %TMP%\mex_%RANDOM%_%TIME:~6,5%_device.obj' ...Yes ('C:\Users\katar\AppData\Local\Temp\mex_25993_49,94_device.obj').
Found installed compiler 'NVIDIA CUDA Compiler'.
Set PATH = C:\Program Files\MATLAB\R2020a\sys\cuda\win64\cuda\nvvm\bin;C:\Program Files\MATLAB\R2020a\sys\cuda\win64\cuda\bin;C:\Program Files\MATLAB\R2020a\sys\cuda\win64\cuda\lib\x64;C:\Program Files\MATLAB\R2020a\sys\cuda\win64\cuda;C:\Program Files\MATLAB\R2020a\bin\win64;C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\Bin\amd64;C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\Bin\VCPackages;C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\..\Common7\IDE;C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\..\Common7\Tools;C:\Program Files (x86)\Windows Kits\8.1\\Bin\x64;C:\Program Files (x86)\Windows Kits\8.1\\Bin\x86;;C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.0\bin;C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.0\libnvvp;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Windows\System32\OpenSSH\;C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;C:\Users\katar\.dnx\bin;C:\Program Files\Microsoft DNX\Dnvm\;C:\Program Files\Microsoft SQL Server\130\Tools\Binn\;C:\Program Files\NVIDIA Corporation\Nsight Compute 2020.1.1\;C:\Program Files\NVIDIA Corporation\NVIDIA NvDLISR;C:\Program Files\MATLAB\R2020a\bin;C:\Program Files\dotnet\;C:\Users\katar\AppData\Local\Microsoft\WindowsApps;C:\Users\katar\.dotnet\tools
Set INCLUDE = C:\Program Files\MATLAB\R2020a\sys\cuda\win64\cuda\include;C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE;C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\ATLMFC\INCLUDE;C:\Program Files (x86)\Windows Kits\10\\include\10.0.18362.0\ucrt;C:\Program Files (x86)\Windows Kits\8.1\\include\shared;C:\Program Files (x86)\Windows Kits\8.1\\include\um;C:\Program Files (x86)\Windows Kits\8.1\\include\winrt;C:\Program Files\MATLAB\R2020a\extern\include;C:\Program Files\MATLAB\R2020a\toolbox\parallel\gpu\extern\include;C:\Program Files\MATLAB\R2020a\sys\cuda\win64\cuda\include;C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE;C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\ATLMFC\INCLUDE;C:\Program Files (x86)\Windows Kits\10\\include\10.0.18362.0\ucrt;C:\Program Files (x86)\Windows Kits\8.1\\include\shared;C:\Program Files (x86)\Windows Kits\8.1\\include\um;C:\Program Files (x86)\Windows Kits\8.1\\include\winrt;C:\Program Files\MATLAB\R2020a\extern\include;C:\Program Files\MATLAB\R2020a\toolbox\parallel\gpu\extern\include;
Set LIB = C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\Lib\amd64;C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\ATLMFC\Lib\amd64;C:\Program Files (x86)\Windows Kits\10\\Lib\10.0.18362.0\ucrt\x64;C:\Program Files (x86)\Windows Kits\8.1\\lib\winv6.3\um\x64;C:\Program Files\MATLAB\R2020a\lib\win64;C:\Program Files\MATLAB\R2020a\sys\cuda\win64\cuda\lib\x64\..;C:\Program Files\MATLAB\R2020a\sys\cuda\win64\cuda\lib\x64;C:\Program Files\MATLAB\R2020a\bin\win64;C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\Lib\amd64;C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\ATLMFC\Lib\amd64;C:\Program Files (x86)\Windows Kits\10\\Lib\10.0.18362.0\ucrt\x64;C:\Program Files (x86)\Windows Kits\8.1\\lib\winv6.3\um\x64;C:\Program Files\MATLAB\R2020a\lib\win64;C:\Program Files\MATLAB\R2020a\sys\cuda\win64\cuda\lib\x64\..;C:\Program Files\MATLAB\R2020a\sys\cuda\win64\cuda\lib\x64;
Set LIBPATH = C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\Lib\amd64;C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\ATLMFC\Lib\amd64;C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\Lib\amd64;C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\ATLMFC\Lib\amd64;
Options file details
-------------------------------------------------------------------
	Compiler location: C:\Program Files\MATLAB\R2020a\sys\cuda\win64\cuda\bin\nvcc.exe
	Options file: C:\Program Files\MATLAB\R2020a\toolbox\parallel\gpu\extern\src\mex\win64\nvcc_msvcpp2015_dynamic.xml
	CMDLINE150 : nvcc -dlink --compiler-options=/Zp8,/GR,/W3,/EHs,/nologo,/MD -gencode=arch=compute_35,code=sm_35 -gencode=arch=compute_50,code=sm_50 -gencode=arch=compute_60,code=sm_60 -gencode=arch=compute_70,code=\"sm_70,compute_70\"  --dont-use-profile -ldir "C:\Program Files\MATLAB\R2020a\bin\win64" --compiler-options=/O2,/Oy-,/DNDEBUG C:\Users\katar\AppData\Local\Temp\mex_5038081698188_12700\mexMPnu8.obj C:\Users\katar\AppData\Local\Temp\mex_5038081698188_12700\c_mexapi_version.obj  -L"C:\Program Files\MATLAB\R2020a\bin\win64" -lcudadevrt -o C:\Users\katar\AppData\Local\Temp\mex_25993_49,94_device.obj && echo /out: > NUL
	CMDLINE200 : link /nologo /manifest  /DLL  /EXPORT:mexFunction /EXPORT:mexfilerequiredapiversion C:\Users\katar\AppData\Local\Temp\mex_25993_49,94_device.obj C:\Users\katar\AppData\Local\Temp\mex_5038081698188_12700\mexMPnu8.obj C:\Users\katar\AppData\Local\Temp\mex_5038081698188_12700\c_mexapi_version.obj  /LIBPATH:"C:\Program Files\MATLAB\R2020a\extern\lib\win64\microsoft" libmx.lib libmex.lib libmat.lib gpu.lib cudart.lib cudadevrt.lib kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib uuid.lib odbc32.lib odbccp32.lib libMatlabDataArray.lib libMatlabEngine.lib /out:mexMPnu8.mexw64
	CMDLINE250 : mt -outputresource:mexMPnu8.mexw64;2 -manifest "mexMPnu8.mexw64.manifest"
	CMDLINE300 : del "mexMPnu8.exp" "mexMPnu8.lib" "mexMPnu8.mexw64.manifest" "mexMPnu8.ilk" C:\Users\katar\AppData\Local\Temp\mex_25993_49,94_device.obj
	COMPILER : nvcc
	COMPFLAGS : --compiler-options=/Zp8,/GR,/W3,/EHs,/nologo,/MD -gencode=arch=compute_35,code=sm_35 -gencode=arch=compute_50,code=sm_50 -gencode=arch=compute_60,code=sm_60 -gencode=arch=compute_70,code=\"sm_70,compute_70\"  --dont-use-profile -ldir "C:\Program Files\MATLAB\R2020a\bin\win64"
	ARCHFLAGS : -gencode=arch=compute_35,code=sm_35 -gencode=arch=compute_50,code=sm_50 -gencode=arch=compute_60,code=sm_60 -gencode=arch=compute_70,code=\"sm_70,compute_70\"  --dont-use-profile -ldir "C:\Program Files\MATLAB\R2020a\bin\win64"
	COMPDEFINES : --compiler-options=/DENABLE_STABLEMODE  --compiler-options=/DMX_COMPAT_64  --compiler-options=/DMATLAB_DEFAULT_RELEASE=R2017b  --compiler-options=/DUSE_MEX_CMD   --compiler-options=/D_CRT_SECURE_NO_DEPRECATE,/D_SCL_SECURE_NO_DEPRECATE,/D_SECURE_SCL=0,/DMATLAB_MEX_FILE
	MATLABMEX : /DMATLAB_MEX_FILE
	OPTIMFLAGS : --compiler-options=/O2,/Oy-,/DNDEBUG
	INCLUDE : -I"C:\Program Files\MATLAB\R2020a\extern\include" -I"C:\Program Files\MATLAB\R2020a\simulink\include" -I"C:\Program Files\MATLAB\R2020a\sys\cuda\win64\cuda\include"
	DEBUGFLAGS : --compiler-options=/Z7
	LINKER : link
	LINKFLAGS : /nologo /manifest 
	LINKTYPE : /DLL
	LINKEXPORT : /EXPORT:mexFunction
	LINKEXPORTVER : /EXPORT:mexFunction /EXPORT:mexfilerequiredapiversion
	LINKLIBS : /LIBPATH:"C:\Program Files\MATLAB\R2020a\extern\lib\win64\microsoft" libmx.lib libmex.lib libmat.lib gpu.lib cudart.lib cudadevrt.lib kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib uuid.lib odbc32.lib odbccp32.lib libMatlabDataArray.lib libMatlabEngine.lib
	NVVMIR_LIBRARY : --dont-use-profile -ldir "C:\Program Files\MATLAB\R2020a\bin\win64"
	LINKDEBUGFLAGS : /debug /PDB:"mexMPnu8.mexw64.pdb"
	LINKOPTIMFLAGS : 
	OBJEXT : .obj
	LDEXT : .mexw64
	SETENV : set COMPILER=nvcc
        set COMPFLAGS=/c --compiler-options=/Zp8,/GR,/W3,/EHs,/nologo,/MD -gencode=arch=compute_35,code=sm_35 -gencode=arch=compute_50,code=sm_50 -gencode=arch=compute_60,code=sm_60 -gencode=arch=compute_70,code=\"sm_70,compute_70\"  --dont-use-profile -ldir "C:\Program Files\MATLAB\R2020a\bin\win64" --compiler-options=/DENABLE_STABLEMODE  --compiler-options=/DMX_COMPAT_64  --compiler-options=/DMATLAB_DEFAULT_RELEASE=R2017b  --compiler-options=/DUSE_MEX_CMD   --compiler-options=/D_CRT_SECURE_NO_DEPRECATE,/D_SCL_SECURE_NO_DEPRECATE,/D_SECURE_SCL=0,/DMATLAB_MEX_FILE /DMATLAB_MEX_FILE
        set OPTIMFLAGS=--compiler-options=/O2,/Oy-,/DNDEBUG
        set DEBUGFLAGS=--compiler-options=/Z7
        set LINKER=link
        set LINKFLAGS=/nologo /manifest  /export:%ENTRYPOINT% /DLL /LIBPATH:"C:\Program Files\MATLAB\R2020a\extern\lib\win64\microsoft" libmx.lib libmex.lib libmat.lib gpu.lib cudart.lib cudadevrt.lib kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib uuid.lib odbc32.lib odbccp32.lib libMatlabDataArray.lib libMatlabEngine.lib /EXPORT:mexFunction
        set LINKDEBUGFLAGS=/debug /PDB:"%OUTDIR%%MEX_NAME%.mexw64.pdb"
        set NAME_OUTPUT=/out:"%OUTDIR%%MEX_NAME%%MEX_EXT%"
	VCROOT : C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC
	SDKROOT : C:\Program Files (x86)\Windows Kits\8.1\
	VSINSTALLDIR : C:\Program Files (x86)\Microsoft Visual Studio 14.0\
	VCINSTALLDIR : C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\
	VCVARSALLDIR : C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC
	KITSROOT : C:\Program Files (x86)\Windows Kits\10\
	SDKVERSION : 10.0.18362.0
	CUDA_LIB_PATH : C:\Program Files\MATLAB\R2020a\sys\cuda\win64\cuda\lib\x64
	CUDA_ROOT : C:\Program Files\MATLAB\R2020a\sys\cuda\win64\cuda
	LIBDEVICE : C:\Program Files\MATLAB\R2020a\bin\win64
	CUDADEVRT : C:\Program Files\MATLAB\R2020a\bin\win64
	DEVOBJ : C:\Users\katar\AppData\Local\Temp\mex_25993_49,94_device.obj
	MATLABROOT : C:\Program Files\MATLAB\R2020a
	ARCH : win64
	SRC : "D:\Kilosort\Kilosort2-master\Kilosort2-master\CUDA\mexMPnu8.cu";"C:\Program Files\MATLAB\R2020a\extern\version\c_mexapi_version.c"
	OBJ : C:\Users\katar\AppData\Local\Temp\mex_5038081698188_12700\mexMPnu8.obj;C:\Users\katar\AppData\Local\Temp\mex_5038081698188_12700\c_mexapi_version.obj
	OBJS : C:\Users\katar\AppData\Local\Temp\mex_5038081698188_12700\mexMPnu8.obj C:\Users\katar\AppData\Local\Temp\mex_5038081698188_12700\c_mexapi_version.obj 
	SRCROOT : D:\Kilosort\Kilosort2-master\Kilosort2-master\CUDA\mexMPnu8
	DEF : C:\Users\katar\AppData\Local\Temp\mex_5038081698188_12700\mexMPnu8.def
	EXP : "mexMPnu8.exp"
	LIB : "mexMPnu8.lib"
	EXE : mexMPnu8.mexw64
	ILK : "mexMPnu8.ilk"
	MANIFEST : "mexMPnu8.mexw64.manifest"
	TEMPNAME : mexMPnu8
	EXEDIR : 
	EXENAME : mexMPnu8
	NVCC_FLAGS : 
	OPTIM : --compiler-options=/O2,/Oy-,/DNDEBUG
	LINKOPTIM : 
	CMDLINE100_0 : nvcc -dc --compiler-options=/Zp8,/GR,/W3,/EHs,/nologo,/MD -gencode=arch=compute_35,code=sm_35 -gencode=arch=compute_50,code=sm_50 -gencode=arch=compute_60,code=sm_60 -gencode=arch=compute_70,code=\"sm_70,compute_70\"  --dont-use-profile -ldir "C:\Program Files\MATLAB\R2020a\bin\win64" --compiler-options=/O2,/Oy-,/DNDEBUG --compiler-options=/DENABLE_STABLEMODE  --compiler-options=/DMX_COMPAT_64  --compiler-options=/DMATLAB_DEFAULT_RELEASE=R2017b  --compiler-options=/DUSE_MEX_CMD   --compiler-options=/D_CRT_SECURE_NO_DEPRECATE,/D_SCL_SECURE_NO_DEPRECATE,/D_SECURE_SCL=0,/DMATLAB_MEX_FILE -I"C:\Program Files\MATLAB\R2020a\extern\include" -I"C:\Program Files\MATLAB\R2020a\simulink\include" -I"C:\Program Files\MATLAB\R2020a\sys\cuda\win64\cuda\include" "D:\Kilosort\Kilosort2-master\Kilosort2-master\CUDA\mexMPnu8.cu" -o C:\Users\katar\AppData\Local\Temp\mex_5038081698188_12700\mexMPnu8.obj && echo /Fo > NUL
	CMDLINE100_1 : nvcc -dc --compiler-options=/Zp8,/GR,/W3,/EHs,/nologo,/MD -gencode=arch=compute_35,code=sm_35 -gencode=arch=compute_50,code=sm_50 -gencode=arch=compute_60,code=sm_60 -gencode=arch=compute_70,code=\"sm_70,compute_70\"  --dont-use-profile -ldir "C:\Program Files\MATLAB\R2020a\bin\win64" --compiler-options=/O2,/Oy-,/DNDEBUG --compiler-options=/DENABLE_STABLEMODE  --compiler-options=/DMX_COMPAT_64  --compiler-options=/DMATLAB_DEFAULT_RELEASE=R2017b  --compiler-options=/DUSE_MEX_CMD   --compiler-options=/D_CRT_SECURE_NO_DEPRECATE,/D_SCL_SECURE_NO_DEPRECATE,/D_SECURE_SCL=0,/DMATLAB_MEX_FILE -I"C:\Program Files\MATLAB\R2020a\extern\include" -I"C:\Program Files\MATLAB\R2020a\simulink\include" -I"C:\Program Files\MATLAB\R2020a\sys\cuda\win64\cuda\include" "C:\Program Files\MATLAB\R2020a\extern\version\c_mexapi_version.c" -o C:\Users\katar\AppData\Local\Temp\mex_5038081698188_12700\c_mexapi_version.obj && echo /Fo > NUL
-------------------------------------------------------------------
Building with 'NVIDIA CUDA Compiler'.
nvcc -dc --compiler-options=/Zp8,/GR,/W3,/EHs,/nologo,/MD -gencode=arch=compute_35,code=sm_35 -gencode=arch=compute_50,code=sm_50 -gencode=arch=compute_60,code=sm_60 -gencode=arch=compute_70,code=\"sm_70,compute_70\"  --dont-use-profile -ldir "C:\Program Files\MATLAB\R2020a\bin\win64" --compiler-options=/O2,/Oy-,/DNDEBUG --compiler-options=/DENABLE_STABLEMODE  --compiler-options=/DMX_COMPAT_64  --compiler-options=/DMATLAB_DEFAULT_RELEASE=R2017b  --compiler-options=/DUSE_MEX_CMD   --compiler-options=/D_CRT_SECURE_NO_DEPRECATE,/D_SCL_SECURE_NO_DEPRECATE,/D_SECURE_SCL=0,/DMATLAB_MEX_FILE -I"C:\Program Files\MATLAB\R2020a\extern\include" -I"C:\Program Files\MATLAB\R2020a\simulink\include" -I"C:\Program Files\MATLAB\R2020a\sys\cuda\win64\cuda\include" "D:\Kilosort\Kilosort2-master\Kilosort2-master\CUDA\mexMPnu8.cu" -o C:\Users\katar\AppData\Local\Temp\mex_5038081698188_12700\mexMPnu8.obj && echo /Fo > NUL
mexMPnu8.cu

nvcc -dc --compiler-options=/Zp8,/GR,/W3,/EHs,/nologo,/MD -gencode=arch=compute_35,code=sm_35 -gencode=arch=compute_50,code=sm_50 -gencode=arch=compute_60,code=sm_60 -gencode=arch=compute_70,code=\"sm_70,compute_70\"  --dont-use-profile -ldir "C:\Program Files\MATLAB\R2020a\bin\win64" --compiler-options=/O2,/Oy-,/DNDEBUG --compiler-options=/DENABLE_STABLEMODE  --compiler-options=/DMX_COMPAT_64  --compiler-options=/DMATLAB_DEFAULT_RELEASE=R2017b  --compiler-options=/DUSE_MEX_CMD   --compiler-options=/D_CRT_SECURE_NO_DEPRECATE,/D_SCL_SECURE_NO_DEPRECATE,/D_SECURE_SCL=0,/DMATLAB_MEX_FILE -I"C:\Program Files\MATLAB\R2020a\extern\include" -I"C:\Program Files\MATLAB\R2020a\simulink\include" -I"C:\Program Files\MATLAB\R2020a\sys\cuda\win64\cuda\include" "C:\Program Files\MATLAB\R2020a\extern\version\c_mexapi_version.c" -o C:\Users\katar\AppData\Local\Temp\mex_5038081698188_12700\c_mexapi_version.obj && echo /Fo > NUL
c_mexapi_version.c

nvcc -dlink --compiler-options=/Zp8,/GR,/W3,/EHs,/nologo,/MD -gencode=arch=compute_35,code=sm_35 -gencode=arch=compute_50,code=sm_50 -gencode=arch=compute_60,code=sm_60 -gencode=arch=compute_70,code=\"sm_70,compute_70\"  --dont-use-profile -ldir "C:\Program Files\MATLAB\R2020a\bin\win64" --compiler-options=/O2,/Oy-,/DNDEBUG C:\Users\katar\AppData\Local\Temp\mex_5038081698188_12700\mexMPnu8.obj C:\Users\katar\AppData\Local\Temp\mex_5038081698188_12700\c_mexapi_version.obj  -L"C:\Program Files\MATLAB\R2020a\bin\win64" -lcudadevrt -o C:\Users\katar\AppData\Local\Temp\mex_25993_49,94_device.obj && echo /out: > NUL
Error using mex
fatbinary fatal   : '94_device.compute_35.cubin' is not in 'keyword=value' format
mexMPnu8.obj
c_mexapi_version.obj



Error in mexcuda (line 166)
    [varargout{1:nargout}] = mex(mexArguments{:});

Error in mexGPUall (line 17)
        mexcuda -v -largeArrayDims -dynamic -DENABLE_STABLEMODE mexMPnu8.cu

If I set set enableStableMode = 0, then it all complies and I am able to run Kilosort without issues.

[jenniferColonell]

Thanks for posting the verbose output!

The issue is that one of the object files in the compile gets automatically named using the system time string:
%RANDOM%_%TIME:~6,5%

On Windows systems using US date/time formats, this will yield a string with only periods, but no commas, e.g.
4729_02.79

On Windows systems using European (and possibly other) formats, commas are inserted into the string -- but commas are not actually allowed in file names, and the file name gets misinterpreted as a compile parameter. We've run into this problem before, specifically when trying to compile using the -dynamic switch. Apparently the standard compile doesn't require that object file to be made.

If you want to be able to run KS2 deterministically on your laptop, you can temporarily change the "Regional Format" to [English (United States)], run the compile, and then switch back.

Running without 'ENABLE_STABLE_MODE' simply means that KS2 will return somewhat different results each time it is run. This can make parameter optimization tricky, because you need to run a few times to see whether a change is within the normal run to run variation.

[Kasia-Bz]

Thank you very much for your help! That solution worked great.