add installation support for pkg-config dependency detection #4077

eli-schwartz · 2022-07-17T16:00:28Z

pkg-config is a build-system agnostic alternative to pybind11Config.cmake that can be used from build systems other than cmake.

Fixes #230

Suggested changelog entry:

* Include a pkg-config file when installing pybind11, such as in the Python package.

eli-schwartz · 2022-07-17T16:02:57Z

tools/JoinPaths.cmake

-    endforeach()
-    set(${joined_path} "${temp_path}" PARENT_SCOPE)
+  set(temp_path "${first_path_segment}")
+  foreach(current_segment IN LISTS ARGN)


Is there some way to tell pre-commit.ci that this file belongs to another repo and shouldn't be autoformatted as that will result in a confusing file delta between here and the upstream version? 👀

Yes, you can add an exclude key regex to the pre-commit-config.yaml for the relevant hook.

Filename seems to have a dot in front of it...

This file is very big and I have basically zero knowledge of how it works other than "frustrating thing that messes up my commit messages", can I have a small hint at what I'm looking at, here? :)

https://github.com/pyodide/pyodide/blob/be1b3c2a6f3ae869300b5630f523dbfb15e3d2e3/.pre-commit-config.yaml#L1

is an example.

Also could be worth mentioning this file can be removed once we depend on CMake 3.20+.

Ah, thanks. Global exclude sounds helpful.

I added comments in CMakeLists.txt, marked with # TODO: for easy searchability.

henryiii · 2022-07-19T08:24:02Z

I'm traveling & at conferences, I'd like to take a look at the generated file. I might want to tweak the CMake code just a bit too. I'll have a bit more time starting Wednesday, so ping me ~Thursday if I don't get back to this by then. Thanks!

eli-schwartz · 2022-07-25T11:56:06Z

Ping. :)

I'd like to take a look at the generated file.

BTW, here is the generated code for the regular wheel:

$ bsdtar -xOf dist/pybind11-2.11.0.dev1-py3-none-any.whl pybind11/share/pkgconfig/pybind11.pc
prefix=${pcfiledir}/../../
includedir=${prefix}/include

Name: pybind11
Description: Seamless operability between C++11 and Python
Version: 2.11.0
Cflags: -I${includedir}

It's the same content for the pybind11_global wheel except, of course, the location.
File lists, more or less:

pybind11:

pybind11/share/cmake/pybind11/FindPythonLibsNew.cmake
pybind11/share/cmake/pybind11/pybind11Common.cmake
pybind11/share/cmake/pybind11/pybind11Config.cmake
pybind11/share/cmake/pybind11/pybind11ConfigVersion.cmake
pybind11/share/cmake/pybind11/pybind11NewTools.cmake
pybind11/share/cmake/pybind11/pybind11Targets.cmake
pybind11/share/cmake/pybind11/pybind11Tools.cmake
pybind11/share/pkgconfig/pybind11.pc

pybind11_global

pybind11_global-2.11.0.dev1.data/data/share/cmake/pybind11/FindPythonLibsNew.cmake
pybind11_global-2.11.0.dev1.data/data/share/cmake/pybind11/pybind11Common.cmake
pybind11_global-2.11.0.dev1.data/data/share/cmake/pybind11/pybind11Config.cmake
pybind11_global-2.11.0.dev1.data/data/share/cmake/pybind11/pybind11ConfigVersion.cmake
pybind11_global-2.11.0.dev1.data/data/share/cmake/pybind11/pybind11NewTools.cmake
pybind11_global-2.11.0.dev1.data/data/share/cmake/pybind11/pybind11Targets.cmake
pybind11_global-2.11.0.dev1.data/data/share/cmake/pybind11/pybind11Tools.cmake
pybind11_global-2.11.0.dev1.data/data/share/pkgconfig/pybind11.pc

eli-schwartz · 2022-07-29T17:14:34Z

@henryiii ping

henryiii · 2022-07-29T18:39:23Z

Thanks! I'm back in Princeton now after nearly a month of travel.

Is there any reason it's joining the path rather than hardcoding the separator, as in ${pcfiledir}/../../? The generated file gets including in the wheel, and must be usable from all OS's (and ideally creatable on all OSs identically), so ${pcfiledir}\..\..\ would be invalid.

eli-schwartz · 2022-07-29T21:46:29Z

The JoinPaths function uses forward slashes anyway, exactly for portability reasons. And the paths need to be joined because cmake's includedir may or may not be an absolute path. (It will be a relative path when invoking cmake from setup.py, though.)

eli-schwartz · 2022-08-01T17:31:28Z

@henryiii the test that you added is failing due to platform differences between \r\n and \n. Not sure what to do about that, I do notice that the pybind11Config.cmake check only tries to assert that a set(...) exists in the file, it's very newline style agnostic.

Is this another of those "setuptools re-encodes all the newlines" issues? :(

eli-schwartz · 2022-08-05T18:25:42Z

By the way, merging this PR would help Void Linux successfully cross-compile SciPy. See scipy/scipy#16783

henryiii · 2022-08-05T18:55:04Z

Sorry, in the third of four conferences in a row. And preparation for CPython 3.10rc1 is taking free time too. This just needs to address the failing test. Not too worried about the line endings, happy to normalize them in the test. I hopefully can do it by Monday, ping me if I don’t! (Or you can do it if you want)

eli-schwartz · 2022-08-05T23:10:33Z

No problem, take your time. I won't have time before then either, anyway. :)

pkg-config is a buildsystem-agnostic alternative to `pybind11Config.cmake` that can be used from build systems other than cmake. Fixes pybind#230

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

rwgk · 2022-08-08T21:22:17Z

This is totally over my head but I looked out of curiosity. Is this impression correct?

pkg-config is build-system agnostic, but the implementation of the pybind11 pkg-config support relies on cmake?

If that's true, it might be nice to add that as a note to the PR description.

Initially the phrase "alternative to pybind11Config.cmake" in the PR description led me to believe the pkg-config support will not involve cmake at all.

eli-schwartz · 2022-08-08T21:35:15Z

CMake is the build system that pybind11 uses, so CMake is used to generate the pkg-config file. Once generated (for example by producing a python wheel, or by running cmake && make install in a Linux distro) the resultant installable files are agnostic.

For the purposes of e.g. SciPy (and more generally meson) we don't involve CMake.

rwgk · 2022-08-08T22:17:36Z

Got it, thanks for the explanation!

eli-schwartz · 2022-08-10T03:00:21Z

@henryiii thanks for figuring out the CI. Really happy to see this landing. :)

eli-schwartz requested a review from henryiii as a code owner July 17, 2022 16:00

eli-schwartz commented Jul 17, 2022

View reviewed changes

eli-schwartz force-pushed the pkgconfig branch from 5413616 to d02bc0c Compare July 17, 2022 18:14

eli-schwartz mentioned this pull request Aug 5, 2022

BUG: meson fails to properly detect numpy, pybind11 and (sort of) pythran when cross compiling scipy/scipy#16783

Closed

henryiii mentioned this pull request Aug 8, 2022

[PLAN] Tracking issue for 2.10.1 #4125

Closed

eli-schwartz and others added 2 commits August 8, 2022 15:40

add installation support for pkg-config dependency detection

c933d9b

pkg-config is a buildsystem-agnostic alternative to `pybind11Config.cmake` that can be used from build systems other than cmake. Fixes pybind#230

tests: add test for pkg config

fc48124

Signed-off-by: Henry Schreiner <henryschreineriii@gmail.com>

henryiii force-pushed the pkgconfig branch from b6a9357 to fc48124 Compare August 8, 2022 19:58

henryiii approved these changes Aug 8, 2022

View reviewed changes

henryiii merged commit 5bdd3d5 into pybind:master Aug 9, 2022

github-actions bot added the needs changelog Possibly needs a changelog entry label Aug 9, 2022

eli-schwartz deleted the pkgconfig branch August 10, 2022 02:59

henryiii removed the needs changelog Possibly needs a changelog entry label Oct 20, 2022

henryiii mentioned this pull request Oct 21, 2022

Scratch PR for testing pybind11 v2.10.0 with Python 3.11rc2 #4271

Closed

rwgk mentioned this pull request Feb 10, 2023

FWD pybind11 google/pybind11k#4077

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add installation support for pkg-config dependency detection #4077

add installation support for pkg-config dependency detection #4077

eli-schwartz commented Jul 17, 2022 •

edited by henryiii

eli-schwartz Jul 17, 2022

Skylion007 Jul 17, 2022

eli-schwartz Jul 17, 2022

henryiii Jul 17, 2022

henryiii Jul 17, 2022 •

edited

eli-schwartz Jul 17, 2022

eli-schwartz Jul 17, 2022

henryiii commented Jul 19, 2022

eli-schwartz commented Jul 25, 2022

eli-schwartz commented Jul 29, 2022

henryiii commented Jul 29, 2022

eli-schwartz commented Jul 29, 2022

eli-schwartz commented Aug 1, 2022 •

edited

eli-schwartz commented Aug 5, 2022

henryiii commented Aug 5, 2022

eli-schwartz commented Aug 5, 2022

rwgk commented Aug 8, 2022

eli-schwartz commented Aug 8, 2022

rwgk commented Aug 8, 2022

eli-schwartz commented Aug 10, 2022

add installation support for pkg-config dependency detection #4077

add installation support for pkg-config dependency detection #4077

Conversation

eli-schwartz commented Jul 17, 2022 • edited by henryiii

Suggested changelog entry:

eli-schwartz Jul 17, 2022

Choose a reason for hiding this comment

Skylion007 Jul 17, 2022

Choose a reason for hiding this comment

eli-schwartz Jul 17, 2022

Choose a reason for hiding this comment

henryiii Jul 17, 2022

Choose a reason for hiding this comment

henryiii Jul 17, 2022 • edited

Choose a reason for hiding this comment

eli-schwartz Jul 17, 2022

Choose a reason for hiding this comment

eli-schwartz Jul 17, 2022

Choose a reason for hiding this comment

henryiii commented Jul 19, 2022

eli-schwartz commented Jul 25, 2022

eli-schwartz commented Jul 29, 2022

henryiii commented Jul 29, 2022

eli-schwartz commented Jul 29, 2022

eli-schwartz commented Aug 1, 2022 • edited

eli-schwartz commented Aug 5, 2022

henryiii commented Aug 5, 2022

eli-schwartz commented Aug 5, 2022

rwgk commented Aug 8, 2022

eli-schwartz commented Aug 8, 2022

rwgk commented Aug 8, 2022

eli-schwartz commented Aug 10, 2022

eli-schwartz commented Jul 17, 2022 •

edited by henryiii

henryiii Jul 17, 2022 •

edited

eli-schwartz commented Aug 1, 2022 •

edited