Using it in a Bazel project

[rickeylev]

Hi,

I'm a owner of rules_python, which is a framework for building Python programs using Bazel.

I suspect that some of our users would be interested in this, so I wanted to give a quick brain dump on the sort of functionality that would make that easier with the constraints Bazel imposes.

The main constraint comes from cross-platform building. The net implication is that you can't run Python itself to get information about Python because the Python runtime to query is for another platform. This, in turn, means that running things like python-config or python -c "import sysconfig; ..." etc to get information about where Python headers are, C ABI settings, etc can't be done.

Instead, with Bazel, this information has to be provided in a platform-neutral format so that Bazel's "setup phase"[1] can transform it into appropriate Bazel configuration information. This setup phase can run arbitrary local programs and Starlark code (Bazel's platform-agnostic DSL).

All of this is to say: if the archive of the runtime includes e.g. a JSON file with information, e.g. where headers are, path to the interpreter binary, python version, abi settings, etc etc), then it can read that and generate the appropriate internal configuration. (I mention JSON because Starlark has builtin support for parsing that, but most any machine readable format can suffice)

What information needs to be provided? The more, the better; it's easy to ignore extra info, harder to reconstruct it after the fact. But a lower bound looks something like:

1. The information needed to copy the runtime somewhere else; the files needed at runtime, e.g. the interpreter, but not necessarily the headers.
2. The information needed to build Python C extensions, e.g. headers, shared libs, abi etc
3. General meta-data, e.g. python version, implementation name, etc

Hope this was helpful / informative!

[geofft]

How much of this information is provided by

• the new standard build-details.json (PEP 739) in Python 3.14+ (n.b. if you're using python-build-standalone, some of the details are slightly incorrect in some of the builds, see cpython#140266 and cpython#136267)
• python-build-standalone's custom PYTHON.json file

?

[brettcannon]

I would be curious to know the answer to these as well, as it's probably better to do this for CPython in general and not specific to this endeavour.

[emmatyping]

Yeah, I know PyO3 has need of this information and has been doing a lot of custom work to get information out of _sysconfigdata. PyO3/pyo3#1672