Build fails with pybind11 2.7

[ianhbell]

I updated the pybind11 module, yielding pybind11 2.7.1.dev1. But the build fails now, with lots of errors of this kind:

CMake Error in pybind11/CMakeLists.txt:
  IMPORTED_LOCATION not set for imported target "pybind11::python_headers"
  configuration "RelWithDebInfo".

They were also present in 2.6, but the build still worked, now it doesn't.

Windows, cmake version 3.19.0-rc2

[henryiii]

I think this is due to the CMake files having absolute paths in the pybind11 2.7.0 package. 2.7.1 is due out today with the fix for this.

[ianhbell]

I see, and the 2.7.1.dev1 doesn't include this fix?

[henryiii]

You mean some commit from master? Depends on which one. And we don't provide wheels for development versions, and "in-place" building should work (it had absolute paths). How are you getting pybind11?

[ianhbell]

I checked out this repo, went into pybind11, and did a pull, which puts me at dcbda8d7ff6e0a207807208ff0364bbb74e02175 for pybind11

[ianhbell]

Then I tried to build this example, no dice

[ianhbell]

with pip install . in the root directory

[henryiii]

Ouch, that's not good. It's also weird, since this is an IMPORTED INTERFACE target, which should not have an IMPORTED_LOCATION - that's only for IMPORTED (no INTERFACE) targets.

[ianhbell]

Yeah, not good indeed. All my pybind11-based projects are going to start failing if/when I update them to this commit.

[ianhbell]

There is more info here: https://cmake.org/cmake/help/latest/policy/CMP0111.html

[ianhbell]

And this is the message with pybind11 2.6:

CMake Warning (dev) in externals/pybind11/CMakeLists.txt:
  Policy CMP0111 is not set: An imported target missing its location property
  fails during generation.  Run "cmake --help-policy CMP0111" for policy
  details.  Use the cmake_policy command to set the policy and suppress this
  warning.

  IMPORTED_LOCATION not set for imported target
  "pybind11::python_link_helper" configuration "RelWithDebInfo".
This warning is for project developers.  Use -Wno-dev to suppress it.

[henryiii]

At least I understand the change in behavior. Pybind11 2.6 used 3.18 as the max policy, and 2.7.1dev1 uses 3.21. But we aren't making any non-INTERFACE targets, and this is working in lots of other places. Are you doing anything special, perhaps? I'll be trying this out on my Windows machine in a minute. So you are doing:

1. Checkout pybind11 master.
2. Install pybind11 into a venv with pip.
3. Checkout CMake example master.
4. Run some sort of command in CMake example? Probably not pip install ., as that will get the latest copy from PyPI when it builds, using PEP 518.

[henryiii]

Oops, no it doesn't, this example uses a submodule, getting them mixed up. I see what you are doing, I'll try it.

[ianhbell]

I never install pybind11 python package. I don't like polluting (and mutating) the packages installed in my environment.

[ianhbell]

I would start from this project, as it very clearly demonstrates the problem. Checkout this project, recursively, update pybind11. Then python setup.py build in the root

[henryiii]

Sorry, I was confused; python_example uses PEP 518 requirements in pyproject.toml, and this one uses a submodule. Eiteh could be used in either example, it was just a choice (as well as assuming that CMake devs might prefer to have it available for building without running the Python parts).

[henryiii]

gh repo clone pybind/cmake_example -- --recurse-submodules
cd cmake_example
git -C pybind11 checkout master
python -m venv .venv
.\.venv\Scripts\python.exe -m pip install -v .

Works just fine, no warnings present. Replacing the last line with an internal detail that should not be exposed, .\.venv\Scripts\python.exe setup.py build, also works. Are you sure you aren't doing something special somewhere? For example, do you have an old version of pybind11's target files cached in your CMake user package registry perhaps?

[henryiii]

PS: I am using a recent CMake instead of 3.19.0rc2 - that's an RC for something that's now on 3.21.1. It's slightly possible this was a regression in that specific version? But unlikely. Claiming an INTERFACE target needs a library location will break a ton of packages.

[henryiii]

Also, very curious, you are getting a IMPORTED target based on that error, but if you are including this via submodule, this is just an INTERFACE target, not an IMPORTED INTERFACE. So I think something unusual is going on. python_link_helper is always an IMPORTED INTERFACE, no INTERFACE only version.

[ianhbell]

Seems like upgrading to cmake 3.21.1 eliminates this particular issue I was seeing. Weird. Running python setup.py build is back in business:

(base) PS C:\XXXXXXX\cmake_example> python setup.py build
running build
running build_ext
-- Building for: Visual Studio 16 2019
-- Selecting Windows SDK version 10.0.18362.0 to target Windows 10.0.18363.
-- The C compiler identification is MSVC 19.29.30038.1
-- The CXX compiler identification is MSVC 19.29.30038.1
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Professional/VC/Tools/MSVC/14.29.30037/bin/Hostx64/x64/cl.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Professional/VC/Tools/MSVC/14.29.30037/bin/Hostx64/x64/cl.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- pybind11 v2.7.1 dev1
-- Found PythonInterp: C:/Users/ihb/Miniconda3/python.exe (found version "3.8.3")
-- Found PythonLibs: C:/Users/ihb/Miniconda3/libs/python38.lib
-- Performing Test HAS_MSVC_GL_LTCG
-- Performing Test HAS_MSVC_GL_LTCG - Success
-- Configuring done
-- Generating done
-- Build files have been written to: C:/Users/ihb/Code/cmake_example/build/temp.win-amd64-3.8/Release
Microsoft (R) Build Engine version 16.10.2+857e5a733 for .NET Framework
Copyright (C) Microsoft Corporation. All rights reserved.

  Checking Build System
  Building Custom Rule C:/Users/ihb/Code/cmake_example/CMakeLists.txt
  main.cpp
     Creating library C:/Users/ihb/Code/cmake_example/build/temp.win-amd64-3.8/Release/Release/cmake_example.lib and object C:/Users/ihb/Code
  /cmake_example/build/temp.win-amd64-3.8/Release/Release/cmake_example.exp
  Generating code
  Finished generating code
  cmake_example.vcxproj -> C:\Users\ihb\Code\cmake_example\build\lib.win-amd64-3.8\cmake_example.cp38-win_amd64.pyd
  Building Custom Rule C:/Users/ihb/Code/cmake_example/CMakeLists.txt

[henryiii]

Great! Probably was due to being a release candidate. If you use pip install . instead, it will download the latest cmake for you using PEP 518, and won't depend on your system CMake. So your bug could have been avoided by not calling setup.py directly, which no one should be doing. pip install or pipx run build are the correct ways to build a Python package.

[ianhbell]

I have a couple of use cases where one can't just use pip install and must build with setup.py. I wish that weren't the case.... But this pipx thing is a new one for me