Linux/cmake build error: libshaderc_combined.a missing and no known rule to create it

[cdwfs]

I'm trying to integrate shaderc into an external project using CMake. In my project's CMakeLists.txt, I have add_subdirectory(third_party/shaderc) to pull in the shaderc tree, and target_link_libraries(my_project_exe shaderc) to add it as a link dependency. This works fine on Windows, but when building on Linux I get an error at build time:
error: '../third_party/shaderc/libshaderc/libshaderc_combined.a', needed by 'third_party/shaderc/libshaderc/CMakeFiles/shaderc_combined_genfile', missing and no known rule to make it
Am I missing a step?

[dneto0]

Shaderc itself depends on multiple libraries, including libshaderc_util, and from SPIRV-Tools and several from Glslang.

The code under shaderc/libshaderc builds two ways:

• shaderc target produces libshaderc.a which then requires you to also supply the other libraries on the link line. That list can change over time (e.g. because Glslang changes)
• shaderc_combined target produces libshaderc_combined.a that pulls in the .o files from the other libraries so your link line is simplified.

I think you want to do:

target_link_libraries(my_project_exe shaderc_combined)

[cdwfs]

No good, I'm afraid; I get the same error as before. Worse, replacing "shaderc" with "shaderc_combined" doesn't work under Windows either.

In the (working) Windows build, I didn't have to manually link against libshaderc's dependencies; just that one target_link_libraries(my_project_exe shaderc) call was sufficient. I see the online-compile example uses this same approach. Does this behavior differ from platform to platform?

[dneto0]

Well I'm glad we've got that example! (Examples > me remembering). I remember now that CMake is smart enough to transitively pull in the dependencies of its targets; that way it's smarter than unix-style linkers.

I verified that I can pull that compile example outside of the shaderc tree, rename its target and build it.

There is a small difference between your example and the compile example. In the compile example it marks the shaderc dependency as private:

add_executable(shaderc-online-compile main.cc)
shaderc_default_compile_options(shaderc-online-compile)
target_link_libraries(shaderc-online-compile PRIVATE shaderc)  ## PRIVATE here

[cdwfs]

Same error again :(

The only thing especially unusual about my setup I can think of is that I'm pulling in shaderc as a git submodule, and since git doesn't seem to allow a project to store submodules inside the trees of other submodules, I have shaderc and all its dependencies living in the same directory:

• third_party/shaderc
• third_party/glslang
• third_party/spirv-tools
• third_party/spirv-headers

This required some extra setup in my CMakeLists.txt:
SET(SPIRV-Headers_SOURCE_DIR "${CMAKE_CURRENT_SOURCE_DIR}/third_party/spirv-headers" CACHE PATH "Location of SPIRV-Headers project" FORCE)
SET(SHADERC_THIRD_PARTY_ROOT_DIR "${CMAKE_CURRENT_SOURCE_DIR}/third_party" CACHE PATH "Root location of shaderc third-party projets" FORCE)
OPTION(SHADERC_SKIP_TESTS "Skip building tests" ON)

Would that make a difference? The error message doesn't look like a problem with shaderc's dependencies, but y'never know...

[dneto0]

I still don't get the error. I replicated with the following CMakeLists.txt file at the top level:

project(cdwfs)

# Check out glslang, spirv-tools, spirv-headers, shaderc into subdirectories of third_party
# Then you should be able to build the foo target.

set(SPIRV-Headers_SOURCE_DIR "${CMAKE_CURRENT_SOURCE_DIR}/third_party/spirv-headers" CACHE PATH "Location of SPIRV-Headers" FORCE)
set(SHADERC_THIRD_PARTY_ROOT_DIR "${CMAKE_CURRENT_SOURCE_DIR}/third_party" CACHE PATH "Root location of Shaderc third party" FORCE)
option(SHADERC_SKIP_TESTS "Skip building tests" ON)

add_subdirectory(third_party/shaderc)

add_executable(foo foo.cc)
target_compile_options(foo PRIVATE -std=c++11)
target_link_libraries(foo shaderc)

And this is foo.cc:

#include <iostream>
#include "shaderc/shaderc.hpp"

int main() {
  shaderc::Compiler c;
  std::cout << "Hello world!" << std::endl;
  return 0;
}

[cdwfs]

Okay, I'll keep poking around. Thanks for confirming I'm not completely off base!

[cdwfs]

Your example worked for me as well, so I started from there and copied things back from my original CMakeLists.txt until it broke again. The culprit was a line I've had in my template forever:

# Remove MinSizeRel and RelWithDebInfo targets
set(CMAKE_CONFIGURATION_TYPES Debug Release CACHE TYPE INTERNAL FORCE )

Adding that to your example is enough to bring back the error I was seeing.

The motivation behind that line was to take the configuration count in Visual Studio from the default 4+ down to the 2 I actually use, but shaderc/cmake/utils.cmake seems to include some logic that changes the library name if it detects more than one configuration type. I'm not sure if that's a legit bug in shaderc or not -- do you expect multi-configuration builds to work on Unix platforms? Either way, it would be easy to tweak the configuration filtering logic on my side to avoid the issue, so that's what I'll do. Thanks again for your help David!

[AWoloszyn]

A quick comment on this, since I think I wrote most of this. As you observed:
We base the location of libshader_combined on where the chosen Generator will put the artefacts.
MSVC and XCode (the only multi-configuration builds that we support) will dump these in Debug/libshaderc_combined.a (or Release/libshader_combined.a) whereas single-configuration generators, (ninja and make) will elide the Release/Debug folders.

So with all of that, we expect that IF you use a multi-configuration builder on *nix platforms, then things should work. Since we determine whether or not you are using a multi-configuration builder by counting CMAKE_CONFIGURATION_TYPES, if it is set for a non multi-configuration generator we do the wrong thing.

So I am also not entirely sure if this is actually a bug or not. I do not know of any multi-configuration generators other than VS and XCode, or if there is any other sane way to determine if we are using such a generator. I also do not know of any other way within cmake to determine if you are using a multi-config generator, rather than enumerating all generators and keeping white/black list with I would prefer not to do, since that is bound to change in the future.

If you happen to know of any more robust way to determine if we are on a multi-config generator I would be happy to hear it.

[cdwfs]

I'm still very much a white belt in terms of CMake-fu, but I think your assessment is correct; I'm not aware of a multi-configuration generator in Unix either, so the failure to handle a forced multi-config build is most likely benign.

I think the bug here was definitely on my side; I should have been more careful messing with CMAKE_CONFIGURATION_TYPES, either by only filtering it when generating for VS or XCode, or using LIST(REMOVE_ITEM) instead of blinding SET()ing the desired values on all platforms. Or just removing that line entirely, because really, the extra configurations aren't hurting anybody.