Build failure on macOS due to Vulkan

[LETIshNick]

I try to build under macos 11.3 (arm) with the latest vulkan 1.2.170.0. To build the source I first evoke cmake . that shows no errors, here is the log.

-- The C compiler identification is AppleClang 12.0.0.12000032
-- The CXX compiler identification is AppleClang 12.0.0.12000032
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found BZip2: /opt/local/lib/libbz2.dylib (found version "1.0.8") 
-- Looking for BZ2_bzCompressInit
-- Looking for BZ2_bzCompressInit - found
-- Found JPEG: /opt/local/lib/libjpeg.dylib (found version "80") 
-- Found ZLIB: /opt/local/lib/libz.dylib (found version "1.2.11") 
-- Looking for fts_set
-- Looking for fts_set - found
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE  
-- Using system zlib, includes found at /opt/local/include
-- Using system jpeg library, includes found at /opt/local/include
-- Using system bzip2 library, includes found at /opt/local/include
-- Looking for strdup
-- Looking for strdup - found
-- Looking for strndup
-- Looking for strndup - found
-- Looking for sys/types.h
-- Looking for sys/types.h - found
-- Looking for stdint.h
-- Looking for stdint.h - found
-- Looking for stddef.h
-- Looking for stddef.h - found
-- Check size of 0i8
-- Check size of 0i8 - failed
-- Check size of 0l
-- Check size of 0l - done
-- Check size of 0ll
-- Check size of 0ll - done
-- Check size of char
-- Check size of char - done
-- Check size of short
-- Check size of short - done
-- Check size of int
-- Check size of int - done
-- Check size of long
-- Check size of long - done
-- Check size of long long
-- Check size of long long - done
-- Check size of void *
-- Check size of void * - done
-- Check size of __int64
-- Check size of __int64 - failed
-- Found PkgConfig: /opt/local/bin/pkg-config (found version "0.29.2") 
-- Found ZMusic: /usr/local/lib/libzmusic.dylib  
Building for target architecture: arm64
-- Performing Test HAVE_MMX
-- Performing Test HAVE_MMX - Failed
-- Performing Test HAVE_PARALLEL_FOR
-- Performing Test HAVE_PARALLEL_FOR - Failed
-- Performing Test HAVE_DISPATCH_APPLY
-- Performing Test HAVE_DISPATCH_APPLY - Success
-- Performing Test HAVE_THREAD_LOCAL
-- Performing Test HAVE_THREAD_LOCAL - Success
-- Looking for filelength
-- Looking for filelength - not found
-- Looking for strupr
-- Looking for strupr - not found
-- Looking for stricmp
-- Looking for stricmp - not found
-- Looking for strnicmp
-- Looking for strnicmp - not found
-- Looking for clock_gettime in rt
-- Looking for clock_gettime in rt - not found
-- Looking for clock_gettime
-- Looking for clock_gettime - found
-- Looking for backtrace
-- Looking for backtrace - found
-- backtrace facility detected in default set of libraries
-- Found Backtrace: /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk/usr/include  
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/lain/Documents/Gaems/GamesWin/doom/gzdoom-master

However when I try to build the target, I have the following error:

[ 30%] Building CXX object src/CMakeFiles/zdoom.dir/common/rendering/vulkan/system/vk_builders.cpp.o
In file included from /Users/lain/Documents/Gaems/GamesWin/doom/gzdoom-master/src/common/rendering/vulkan/system/vk_builders.cpp:27:
In file included from /Users/lain/Documents/Gaems/GamesWin/doom/gzdoom-master/src/../libraries/glslang/spirv/GlslangToSpv.h:42:
In file included from /Users/lain/Documents/Gaems/GamesWin/doom/gzdoom-master/src/../libraries/glslang/spirv/SpvTools.h:49:
/usr/local/include/glslang/MachineIndependent/localintermediate.h:406:14: error: 
      use of undeclared identifier 'EShTargetVulkan_1_2'
        case EShTargetVulkan_1_2:
             ^
/usr/local/include/glslang/MachineIndependent/localintermediate.h:421:28: error: 
      use of undeclared identifier 'EShLangRayGen'; did you mean
      'EShLangRayGenNV'?
        return language >= EShLangRayGen && language <= EShLangCallableNV;
                           ^~~~~~~~~~~~~
                           EShLangRayGenNV
/Users/lain/Documents/Gaems/GamesWin/doom/gzdoom-master/src/../libraries/glslang/glslang/Public/ShaderLang.h:95:5: note: 
      'EShLangRayGenNV' declared here
    EShLangRayGenNV,
    ^
2 errors generated.

[alexey-lysiuk]

GZDoom doesn’t need Vulkan SDK in order to compile it. Moreover, we only support building with glslang stored inside GZDoom repository. I have no idea how you managed to mix two glslang versions, from /usr/local and from GZDoom itself. All I know that it compiles just fine without Vulkan SDK installed.

[LETIshNick]

Well, I did nothing special, just followed the instructions. I have vulkan installed because I built vkquake recently.

[alexey-lysiuk]

What are those instructions? What’s the version of Vulkan SDK? Is it a native build or cross-compilation? Nobody is able to help you as you provided almost no information.

[LETIshNick]

Excuse me, I thought that the problem could be resolved at first glance. Updated the first post.

[alexey-lysiuk]

Please post output of make VERBOSE=1, and attach CMakeCache.txt file from build directory.

[Cacodemon345]

I faced this problem before but on Linux. The CMake script does not add the glslang include directory to the list of include search paths so it will prefer the system's glslang include directory if it is found, otherwise falling back to the one that is bundled with the project.

[alexey-lysiuk]

Why didn’t you report this then?

[Cacodemon345]

Because I forgot after I removed system glslang.

[LETIshNick]

@alexey-lysiuk anyways, here is the output, hope this helps.
https://gist.github.com/LETIshNick/dd23a2c34904d6d9010a6631636cdd9a

[alexey-lysiuk]

In theory, this change should fix any kind of internal and system include paths mixing.

git checkout -b reorder_include_directories master
git pull https://github.com/alexey-lysiuk/gzdoom.git reorder_include_directories

I don't want to push such complications to master branch directly without a confirmation that it fixes the issue in the affected environment.

[LETIshNick]

This one fails with a completely unrelated error, https://gist.github.com/LETIshNick/98831494d5e38ecc87e7866572e0cb26

[alexey-lysiuk]

Certainly, you did something wrong. Most likely, you checked out a wrong branch. Dumb library that causes compilation failure is not a part of GZDoom repository since 4.4.0 while my branch is a few commits older than 4.6.0 release.

[coelckers]

I faced this problem before but on Linux. The CMake script does not add the glslang include directory to the list of include search paths so it will prefer the system's glslang include directory if it is found, otherwise falling back to the one that is bundled with the project.

I just noticed that, too., while checking out some things in the CMake project.
Makes me wonder how it still manages to compile then. My main problem is that due to compilation still working I cannot see what is wrong here

So what do we do here that it properly sets these up?

[alexey-lysiuk]

Well, if "do not install random crap in your system" is not an option, I propose to reorder include directories to place internal paths before external. alexey-lysiuk@5c9fb20. Need a volunteer to test it on affected system.

[LETIshNick]

Having my thesis finished, I can do some testing now.