Cmake with add subdirectory

[mathstuf]

---

Resubmission of #967 with better commit messages.

Cc: @tjcorona

[tjcorona]

Would someone mind reviewing this PR?

[jagerman]

Can you give me an example parent project CMakeLists.txt where this fixes an issue/allows a feature? (Ideally we'd add a test for it, but that might be tricky). I'm afraid my understanding of CMake is a bit limited w.r.t this PR, but I suspect a good demonstrating example would help me out a lot.

Cc: @dean0x7d whose CMake understanding is much better than mine.

One other note about this PR (for now)--it'll need a fix for the pypy issue. (I think it would be enough to add -C / to the tar arguments that extract pypy, and to simplify the next line to just PY_CMD=/pypy2-v5.8.0-linux64/bin/pypy. (The existing use of a subshell invoking echo there is a bit peculiar; no need to preserve it).

[dean0x7d]

The BUILD_INTERFACE for Python includes and libraries is needed because we don't want to hardcode a specific Python version/distribution with the installed pybind11. That would require multiple pybind11 installations -- one for each Python variant. Instead, we defer Python selection to configuration time and append the required properties in pybind11Config.cmake.

So BUILD_INTERFACE definitely needs to stay as it is. Reverting that commit would also resolve the error on Travis.

Regarding the second part, I assume the intended usage from the point of view of the parent project is something like:

set(PYBIND11_INSTALL ON)
set(PYBIND11_INSTALL_EXPORT OFF)
add_subdirectory(pybind11)
install(EXPORT pybind11Targets ...)

Is this correct?

What's the intended usage for PYBIND11_EXPORT_NAME? It seem like the install(EXPORT <export-name> FILE <name>.cmake) command should make it possible to rename the export. We'd like to keep the number of configuration variables to a minimum. If this works for you, removing PYBIND11_EXPORT_NAME would be preferable.

[mathstuf]

The BUILD_INTERFACE for Python includes and libraries is needed because we don't want to hardcode a specific Python version/distribution with the installed pybind11.

Ah, that makes sense. However, that doesn't work for projects which are embedding Pybind11 though; they're using it incidentally and missing the required headers since they don't do find_package(Pybind11), but are instead just getting the targets from a find_package(UsesPybind11) package.

Is this correct?

Sort of. The idea is to just flag it based on PYBIND11_EXPORT_NAME instead. The name needs to be the same because it's an error in CMake to install a library as part of an export set that doesn't contain all of the dependencies (that are built by the project).

For example, this fails without being able to change the export name of pybind11 (when placed in tests/cmake_test_embed_source):

cmake_minimum_required(VERSION 2.8.12)                                                                                                                                                                                                                                                                                        
project(test_embed)

add_subdirectory("${CMAKE_SOURCE_DIR}/../.." pybind11)

add_library(test_embed
  test_embed.cpp)
target_link_libraries(test_embed
  pybind11::embed)
install(
  TARGETS test_embed
  EXPORT  test_export
  ARCHIVE DESTINATION bin
  LIBRARY DESTINATION lib
  RUNTIME DESTINATION lib)
install(
  EXPORT      test_export
  DESTINATION lib/cmake/test_export/test_export-Targets.cmake)

test_embed.cpp is:

#include <pybind11/embed.h>

int foo() {
    return 0;
}

With this error:

CMake Error: install(EXPORT "test_export" ...) includes target "test_embed" which requires target "embed" that is not in the export set.

[mathstuf]

For completeness, the test suite would build and install that project and then have another project use the install of the first project (which is where BUILD_INTERFACE shows itself as being a problem).

[dean0x7d]

However, that doesn't work for projects which are embedding Pybind11 though; they're using it incidentally and missing the required headers since they don't do find_package(Pybind11), but are instead just getting the targets from a find_package(UsesPybind11) package.

The solution here should be for the parent package (UsesPybind11) to configure Python in its UsesPybind11Config.cmake file, either by doing find_package(pybind11) or by directly including pybind11Config.cmake. This is a bit more work for the parent package's config file, but don't see any way around it. We must defer Python selection to configuration time. Otherwise, the packaging/Python variants become unmanageable.

The idea is to just flag it based on PYBIND11_EXPORT_NAME instead. The name needs to be the same because it's an error in CMake to install a library as part of an export set that doesn't contain all of the dependencies (that are built by the project).

Oh, I see. I missed the fact that this would append pybind11's target to the existing export set. OK, this looks fine to me then.

[dean0x7d]

As mentioned above, the BUILD_INTERFACE changes need to be reverted. The parent project's config file will need to call pybind11's Python config (transparent for end users).

The export name solution looks good to me. I just left some minor organizational comments regarding the new option variables.

[dean0x7d]

This should be an option() just like PYBIND11_INSTALL below.

[mathstuf]

It's not really a user-facing option, but a project level option; it should not be in the cache.

[dean0x7d]

If that's the case (it shouldn't be configurable at all), it looks like if(PYBIND11_INSTALL_EXPORT) could simply be replaced by if(PYBIND11_MASTER_PROJECT) and PYBIND11_INSTALL_EXPORT could be removed altogether.

[dean0x7d]

Since PYBIND11_EXPORT_NAME is only ever needed in the if(PYBIND11_INSTALL) block, I'd move it in there to improve locality. No need for separate PYBIND11_MASTER_PROJECT behavior.

[mathstuf]

OK.

[mathstuf]

The thing is that we're installing a vendored Pybind11; there isn't really a Pybind11 to find at all. Since it's embedded, we have no need for flexibility with the installed Python either. Is it fine if the BUILD_INTERFACE is conditional?

[dean0x7d]

Is it fine if the BUILD_INTERFACE is conditional?

I would very much like to avoid that if at all possible. Even if the config file can't find_package(pybind11), it should still be able to do the equivalent by including the relevant lines to populate the Python-specific properties.

[mathstuf]

OK, I removed the BUILD_INTERFACE bit.

[dean0x7d]

Merged, thanks!