ENH: add tool.meson-python.wheel.exclude setting

[dnicolodi]

I am not entirely sure we want to include this facility: it may potentially be abused when better solutions exist. However, I think it is better than the solution we currently recommend https://mesonbuild.com/meson-python/how-to-guides/shared-libraries.html#shared-library-from-subproject of adding specific options to subprojects to exclude this or that parts from the installation.

[dnicolodi]

Another idea I was toying with is to extend the filtering by meson install tags to allow to specify different tags per subproject. meson-python does not rely on the meson implementation of the filtering anyway, thus extending the facility is possible.

[rgommers]

Thanks @dnicolodi! I like this. Agreed that these unmappable install locations/targets are a pain point, and this is a nice and ergonomic solution I think.

The wheel.exclude name is good, and it matches the name in scikit-build-core which is helpful.

One question about using introspection metadata locations. I can see that working, but there's also an alternative of using wheel locations - I'm not quite sure which is better.

[rgommers]

Another idea I was toying with is to extend the filtering by meson install tags to allow to specify different tags per subproject. meson-python does not rely on the meson implementation of the filtering anyway, thus extending the facility is possible.

That may still require changes to the subproject itself (e.g., there are multiple shared libraries and you need only one of them), so is less general. I prefer wheel.exclude because of that.

[dnicolodi]

We could also consider implementing both wheel.exclude and wheel.include so that the first could be used to exclude a whole subtree and the latter to include selected files in the same subtree.

[rgommers]

We could also consider implementing both wheel.exclude and wheel.include so that the first could be used to exclude a whole subtree and the latter to include selected files in the same subtree.

I can see the appeal of that, yes. Especially on projects with lots of install targets, that seems nice. Although the cases where you can't make things concise with excluding install tags + wheel.exclude may be a small minority.

This could be done immediately, or later if there's demand. Given this PR is very nice and clean, may be okay to add it immediately?

[dnicolodi]

I've implemented tool.meson-python.wheel.include too. I think this should be ready now.

Working on this it came to my mind that we could also have a tool.meson-python.wheel.map that allows to map files installed in one location to another. I'm just worried that it may be misused.

[rgommers]

Working on this it came to my mind that we could also have a tool.meson-python.wheel.map that allows to map files installed in one location to another. I'm just worried that it may be misused.

Yeah potentially useful, but also potentially easy to misuse. This thread from yesterday is a good example: https://discuss.python.org/t/include-header-above-project-root/102221. Could potentially be a nicer way to include a C library that's one folder up in the repo, instead of symlinking it in - we've had examples of this, like nanoarrow.

On the other hand, this seems quite bad: https://hatch.pypa.io/latest/config/build/#forced-inclusion. It gives an explicit example of pulling in a header file from outside the repo, which seems like a good way to create non-reproducible builds/issues.

[eli-schwartz]

Hatchling has had some really quite inspirational bugs if I recall correctly, related to trying to match inclusion patterns against parent directories of the project root. Very fun, in the sense of not being fun at all, when it incorrectly operates on the standard system directory structure used for ALL package builds.

Could potentially be a nicer way to include a C library that's one folder up in the repo, instead of symlinking it in - we've had examples of this, like nanoarrow.

Not really sure I understand the problem. If you're building a C library one folder up in the repo, I'm guessing this is a C library and its optional bindings. You really want the entire project to be a single build including both C and python bits. Autotools / Meson / CMake can all do that already, with varying abilities to install the python code in the right site-packages directory, and with meson-python you can even support both building a system C library and its bindings, OR a PyPI wheel that statically links the C library for standalone distribution.

You really don't want to require "one folder up" because it's not possible to produce an sdist like that, and you can't guarantee the C library is built when needed if it's two separate builds.

[dnicolodi]

Yeah potentially useful, but also potentially easy to misuse. This thread from yesterday is a good example: https://discuss.python.org/t/include-header-above-project-root/102221. Could potentially be a nicer way to include a C library that's one folder up in the repo, instead of symlinking it in - we've had examples of this, like nanoarrow.

What I had in mind would not solve this issue. What I had in mind is a facility that maps a Meson installation path, for example to another, for example {py_platlib}/package/foolib.so to {py_platlib}/package/foolib.so. However, just while writing this I realized that a simple mapping like this is not going to be easy to use as the filename to be mapped is platform dependent.

[rgommers]

Not really sure I understand the problem. If you're building a C library one folder up in the repo, I'm guessing this is a C library and its optional bindings. You really want the entire project to be a single build including both C and python bits. Autotools / Meson / CMake can all do that already,

The point is that you can't do it through a standard build frontend interface if the C library code isn't inside the part of the repo containing pyproject.toml. gh-674 is the example I was thinking about, there have been others. The problem to solve is how to invoke python -m build --wheel or similar and pull in that C library.

What I had in mind would not solve this issue.. [...] I realized that a simple mapping like this is not going to be easy to use as the filename to be mapped is platform dependent.

Good point. Okay, leaving that problem for another day then:)

The added include part also looks good on a first read. Let me see if I can test and hit the green button on this one after dinner.

[rgommers]

However, I think it is better than the solution we currently recommend https://mesonbuild.com/meson-python/how-to-guides/shared-libraries.html#shared-library-from-subproject of adding specific options to subprojects to exclude this or that parts from the installation.

I assume you want to leave an update to those docs out of this PR?

[dnicolodi]

I've updated the documentation.

[rgommers]

The docs LGTM, thanks. Two small suggestions only. Also there's a merge conflict to resolve. When that is resolved, this looks ready to go in.