Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Python bindings build using bzlmod #1764

Merged
merged 3 commits into from
Mar 7, 2024

Conversation

nicholasjng
Copy link
Contributor

Uses the newly started @nanobind_bazel project to build nanobind extensions. This means that we can drop all in-tree custom build defs and build files for nanobind and the C++ Python headers.

Additionally, the temporary WORKSPACE overwrite hack naturally goes away due to the WORKSPACE system being obsolete.


The git override might be a little shaky, but I have not released nanobind-bazel on the BCR yet, so this will have to do for now as a proof of concept.

Backstory & explanations

Before, google_benchmark had a custom Python binding extension in-tree (bindings/python/google_benchmark/build_defs.bzl), which was defined back in the pybind11 days (and possibly even predating pybind11_bazel, which also contains these defs).

Since bindings are now built using nanobind, I decided to make use of nanobind-bazel, a self-developed counterpart for easy building of nanobind extensions with Bazel. Its main strength is that it also includes flags for optionally building with size optimizations enabled, targeting the Python stable ABI, and platform-specific compiler and linker options.

The main benefit for GBM is that all of these aspects are now in nanobind_bazel's hands, which relieves maintainers from having to adjust those optimizations (and if necessary, implement them as patch files to use in a git_override).

Another reason why this has become feasible now is that instead of having to monkey-patch the Python include path, newer versions of @rules_python include the @rules_python//python/cc:current_py_cc_headers target, which gives the header files for the currently configured Python toolchain. The targeted Python interpreter can be given via a build flag, which is done in the build command in setup.py.

For our (admittedly clunky) way of building the Python wheels, this means that the .so files are built with a hermetic interpreter, which looks a bit sketchy, but ultimately does not break anything as the C API is promised stable in a given minor version of Python.

Currently, the Python stable ABI is targeted starting from Python 3.12, which means that Python users of Google Benchmark will enjoy stability across Python minor versions going forward. A quick audit of the _benchmark.so extension on my local machine shows no ABI violations, but adding such a check into CI might be a good idea.

Uses the newly started `@nanobind_bazel` project to build nanobind
extensions. This means that we can drop all in-tree custom build defs
and build files for nanobind and the C++ Python headers.

Additionally, the temporary WORKSPACE overwrite hack naturally goes away
due to the WORKSPACE system being obsolete.
The latest minor releases incurred some formatting and configuration
changes, this commit rolls them out.
@nicholasjng
Copy link
Contributor Author

@macandy13 I would of course be happy to hear your opinions here if you're still interested.

@dmah42
Copy link
Member

dmah42 commented Mar 7, 2024

thanks for the comprehensive PR description! i'll merge this as soon as CI passes with the merge from main.

@dmah42 dmah42 merged commit eaafe69 into google:main Mar 7, 2024
80 checks passed
@nicholasjng
Copy link
Contributor Author

Thank you! This should also conclude the bzlmod migration :)

@nicholasjng nicholasjng deleted the add-bzlmod-bindings branch March 7, 2024 12:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants