feat(py): Python interpreter provisioning from python-build-standalone by arrdem · Pull Request #827 · aspect-build/rules_py

arrdem · 2026-03-06T19:42:56Z

Adds automatic Python interpreter provisioning from
python-build-standalone
(PBS), replacing the need for users to manually specify interpreter URLs and
checksums or rely on rules_python's version manifests.

Interpreters are discovered automatically from PBS release artifacts and cached
in MODULE.bazel.lock via the Bazel facts API. The root module owns all
configuration — release dates, mirror URLs, pre-release policy, default version
— while dependency modules can request Python versions additively.

Usage

interpreters = use_extension("@aspect_rules_py//py/unstable:extension.bzl", "python_interpreters")
interpreters.configure(
    releases = ["20260303", "20241002"],
)
interpreters.toolchain(python_version = "3.13", is_default = True)
interpreters.toolchain(python_version = "3.11")
interpreters.toolchain(python_version = "3.15", pre_release = True)

use_repo(interpreters, "python_interpreters")
register_toolchains("@python_interpreters//...")

Module scoping

Setting	Root module	Non-root module
`configure()`	Sets release search space and mirror	Silently ignored
`toolchain(python_version)`	Adds to global set	Adds to global set
`toolchain(is_default)`	Honored	Silently ignored
`toolchain(pre_release)`	Honored	Silently ignored

Changes are visible to end-users: yes

Searched for relevant documentation and updated as needed: yes
Breaking change (forces users to change their own code or config): no
Suggested release notes appear below: yes

New python_interpreters module extension at //py/unstable:extension.bzl for
automatic Python interpreter provisioning from python-build-standalone. Supports
release-date-based versioning, facts API caching, freethreaded builds,
pre-release version policies, and root-module-scoped configuration.

Test plan

New test cases added
bazel test //... — version_util unit tests
cd e2e && bazel test //... — freethreaded e2e test, interpreter provenance test
e2e/cases/interpreter-provisioning verifies sys.executable comes from python_3_* repos, not rules_python

CLAassistant · 2026-03-06T19:43:03Z

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 3 committers have signed the CLA.

✅ arrdem
❌ claude
❌ aspect-marvin
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

aspect-workflows · 2026-03-06T19:44:03Z

Bazel 8 (Test)

All tests were cache hits

103 tests (100.0%) were fully cached saving 46s.

Bazel 9 (Test)

All tests were cache hits

103 tests (100.0%) were fully cached saving 1m 14s.

Bazel 8 (Test)

e2e

1 test target passed

Targets

//cases/interpreter-provisioning:test [k8-fastbuild-ST-1a8602e72ef0]180ms

Total test execution time was 180ms. 28 tests (96.6%) were fully cached saving 25s.

Bazel 9 (Test)

e2e

1 test target passed

Targets

//cases/interpreter-provisioning:test [k8-fastbuild-ST-1a8602e72ef0]132ms

Total test execution time was 132ms. 28 tests (96.6%) were fully cached saving 25s.

Bazel 8 (Test)

examples/uv_pip_compile

All tests were cache hits

1 test (100.0%) was fully cached saving 444ms.

…alone Adds a module extension `python_interpreters` that downloads CPython interpreters directly from astral-sh/python-build-standalone releases, bypassing rules_python for interpreter provisioning. New files: - py/interpreter.bzl — public API entry point - py/private/interpreter/versions.bzl — PBS release metadata (3.11, 3.12, 3.13) - py/private/interpreter/repository.bzl — repository rules for downloading interpreters and creating toolchain registrations - py/private/interpreter/extension.bzl — module extension implementation - py/private/interpreter/BUILD.bazel — python_version string flags Usage in MODULE.bazel: interpreters = use_extension("@aspect_rules_py//py:interpreter.bzl", "python_interpreters") interpreters.toolchain(python_version = "3.12", is_default = True) interpreters.toolchain(python_version = "3.13") use_repo(interpreters, "python_interpreters") register_toolchains("@python_interpreters//:all") Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Updates the transition to set both our own python_version flag (@aspect_rules_py//py/private/interpreter:python_version) and the rules_python flag for backward compatibility. Both are kept in sync during the migration period. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Replace rules_python python.toolchain() with our own interpreters.toolchain() in e2e/MODULE.bazel. Add Python 3.9 and 3.10 to versions.bzl. Add version-conditioned toolchain resolution via config_settings and target_settings so the correct interpreter is selected when python_version is set. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Replace hardcoded version/SHA256 tables with a release-date-based architecture that discovers interpreter versions and checksums at repository-rule time from the PBS SHA256SUMS file. Key changes: - versions.bzl now contains only a minimal mapping of release dates to available minor versions (~20 lines), plus platform and build config definitions. No SHA256 hashes or platform-specific data. - Repository rules download SHA256SUMS (~100KB) to discover the exact patch version and checksum for each interpreter, then download the matching archive. - Module extension supports a new `release` tag for specifying PBS release dates, with sensible defaults covering Python 3.8-3.15. - Version routing prefers the newest release date for each version. - Platforms expanded to include Windows (x86_64, aarch64, i686) and linux-musl (aarch64). - Unavailable version/platform combinations generate stub BUILD files instead of failing, so toolchain resolution gracefully skips them. - New `freethreaded` bool flag and `build_config` string flag for selecting interpreter build configurations. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Move SHA256SUMS fetching from repository rules into the module extension using module_ctx.download(). The parsed release indices are cached in MODULE.bazel.lock via the Bazel facts API (module_ctx.facts / extension_metadata(facts=...)), so subsequent builds skip the downloads. This eliminates the RELEASES mapping entirely — versions.bzl now contains only default release dates (a flat list of 3 strings), platform constraint mappings, and build config definitions. No version-to-release routing table needed; the extension discovers everything from SHA256SUMS. Repository rules are now simple: they receive a pre-resolved URL and SHA256 from the extension and just download + extract. Unavailable version/platform combos get an empty URL and generate a stub BUILD. Falls back gracefully when the facts API is unavailable (Bazel < 8.x) by re-downloading SHA256SUMS each time. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Two new features for the interpreter provisioning extension: 1. interpreters.release(date = "latest") resolves to the newest PBS release via the GitHub releases API. This marks the extension as non-reproducible (reproducible = False in extension_metadata), so Bazel re-evaluates it on each invocation. The resolved datestamp is still cached in facts under its real date, so only the "latest" resolution is non-reproducible — the actual release data is cached. 2. interpreters.release(base_url = "...") allows overriding the PBS download URL, enabling fetches from mirrors or forks. The base_url is propagated through to all interpreter downloads from that release. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Covers quickstart, release dates, "latest" resolution, mirrors, build configs, per-target version selection, platforms, and rules_python compatibility. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Replace the constraint_setting/constraint_value pairs for interpreter feature flags (pydebug, pymalloc, freethreading, wide_unicode) with config_setting targets backed by bool_flags in //py/private/interpreter. This bridges the interpreter toolchain provisioning system with the uv wheel selection system — setting --@aspect_rules_py//py/private/interpreter:freethreaded=true now correctly influences both toolchain resolution AND ABI-tagged wheel selection. Also adds pydebug, pymalloc, wide_unicode bool_flags alongside the existing freethreaded flag, and removes the unused python_version_major_minor string_flag. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

… path (#805) Freethreaded Python (3.13t+) uses `lib/python3.13t/site-packages` instead of `lib/python3.13/site-packages`. The venv tool was hardcoding the non-freethreaded path, causing `ModuleNotFoundError` for any packages installed into a freethreaded venv. - Add `--freethreaded` flag to the venv Rust tool - Add `freethreaded` field to `PythonVersionInfo` with `lib_suffix()` - Read `//py/private/interpreter:freethreaded` bool_flag in py_venv rule - Fix `_sanitize()` to handle `+` in build config names - Add e2e test case (cases/freethreaded-805) that verifies: - Interpreter reports `Py_GIL_DISABLED=1` - SOABI contains the `t` suffix - `regex` native extension imports and works - The `.so` filename contains `cpython-313t` - Register `freethreaded+pgo+lto` 3.13 toolchain in e2e Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

The interpreter extension now registers ALL build configs (install_only, install_only_stripped, freethreaded+pgo+lto, freethreaded+debug) for each requested Python version automatically. Users select configs via flags or custom platform() targets rather than declaring build_config in MODULE.bazel. The freethreaded e2e test now uses platform_transition_test from bazel_lib with a custom platform that sets --freethreaded=true, so it runs automatically without manual flags. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Linux glibc and musl platforms share the same constraint_values (os:linux + cpu arch), making them ambiguous to Bazel's toolchain resolution. This adds target_settings that match against the existing platform_libc string_flag from the uv extension, so each Linux toolchain is only selected when the libc type matches. - versions.bzl: Add target_settings to glibc/musl PLATFORMS entries - extension.bzl: Pass platform_target_settings through toolchain JSON - repository.bzl: Generate config_setting targets in the hub repo and wire them into each toolchain's target_settings Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Every platform now consistently declares its libc type via target_settings: libsystem (macOS), msvc (Windows), glibc/musl (Linux). This ensures interpreter toolchains are always disambiguated by the platform_libc flag, not just on Linux. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

If _parse_sha256sums returns an empty index (e.g. typo in release date producing an HTML error page or an unrecognized format), fail() with a clear message pointing the user to the PBS releases page. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

…egistering stubs Instead of creating stub repos with sentinel config_settings for version/platform/config combinations that don't exist in any PBS release, simply skip them. This prunes the toolchain matrix to only real assets, reducing the number of registered toolchains and removing the _UNAVAILABLE_BUILD template entirely. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

…uilds If a user requests a Python version (e.g. "3.99") that doesn't exist in any configured PBS release, fail() immediately with a clear message listing the release dates searched, rather than silently producing no toolchains. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

All build configs are registered automatically; there's nothing to select via the tag class. Remove the dead attr. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

…eature flag Replaces the four individual bool_flags (freethreaded, pydebug, pymalloc, wide_unicode) with a single repeatable --interpreter_feature string flag. The pattern follows exclude_feature from #836: a user-facing allow_multiple flag, with derived interpreter_has_feature rules that read the list and expose FeatureFlagInfo("true"/"false"). This enables both presence and absence matching via config_setting flag_values, which is required for correct ABI tag anti-matching and toolchain selection. Usage: --@aspect_rules_py//py/private/interpreter:interpreter_feature=freethreaded Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

…preter_feature flag" The derived-flag approach (interpreter_has_feature exposing FeatureFlagInfo) doesn't work with config_setting.flag_values in toolchain target_settings. Bazel's config_setting reads the raw build_setting_value, not the provider returned by the implementation. Reverts to individual bool_flags which work correctly for both toolchain selection and ABI tag anti-matching. Also removes the orphaned build_config string_flag. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

…s cache key Two bugs found during review: 1. The python_transition didn't include the freethreaded bool_flag in its inputs/outputs, so dependencies of py_binary/py_venv targets using the transition would always see the default (False) regardless of the user's setting. The e2e test passed because it uses platform_transition_test which bypasses python_transition. 2. The facts cache key for release indices only included the release date, not the base_url. Two releases with the same date but different base_urls (e.g. mirror vs upstream) would return stale cached data. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

…licy - Extract version comparison into version_util.bzl with proper PEP 440 pre-release handling (alpha < beta < rc < release). The old _version_gt crashed on versions like 3.15.0a6 because int("0a6") fails. - Add `pre_release` attr to the toolchain tag (default False). When False, pre-release versions are skipped so requesting python_version = "3.16" won't silently serve 3.16.0a2. - Add Starlark unit tests covering version_key, version_gt, is_pre_release, pre-release ordering, and padding behavior. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

…elease Specifying python_version = "3.15.0a2" now implicitly sets pre_release = True for that major.minor, since the user's intent is unambiguous. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Move the public API for python_interpreters from py/interpreter.bzl to py/unstable/extension.bzl, matching the uv/unstable/extension.bzl pattern. Delete the top-level py/interpreter.bzl facade. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

…lass Consolidate release date configuration into a single configure() tag with a string_list attribute, and enforce module scoping rules: - configure() is silently ignored from non-root modules - is_default and pre_release on toolchain() are root-module-only - Any module can request versions via toolchain() - Error messages now identify which module requested a missing version Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Add an e2e test that asserts sys.executable comes from our python_3_* repos, not from rules_python's pythons_hub. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Cover the full version selection story: default toolchain, .bazelrc flag as a build-wide default, command-line override for ad-hoc testing, and per-target python_version attribute. Include precedence table. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Rename //py:interpreter_version to //py:python_version for clarity. Update docs and e2e test .bazelrc references. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Clarify that uv.project() has no dependency on any particular interpreter provisioning mechanism — it only needs a registered Python toolchain at build time. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

sys.executable in a py_venv_test points to the venv's bin/python symlink, not the underlying interpreter repo path. Use os.path.realpath() to resolve it and simplify to just the negative assertion (not rules_python). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

arrdem marked this pull request as ready for review March 7, 2026 00:46

arrdem force-pushed the arrdem/feat-python-interpreters branch 2 times, most recently from 2082491 to 35b96b3 Compare March 7, 2026 05:41

claude and others added 25 commits March 10, 2026 09:21

chore(py): Point interpreter_version alias to our own flag

c1950c4

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Pre-commit changes

183b45a

Pre-commit changes

6117f81

Pre-commit changes

3f5c1fa

Pre-commit changes

68f09de

docs(py): Add interpreter provisioning documentation

3fcd675

Covers quickstart, release dates, "latest" resolution, mirrors, build configs, per-target version selection, platforms, and rules_python compatibility. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Pre-commit changes

ac4ac41

chore(py): Remove unused build_config attr from toolchain tag

fa9ff66

All build configs are registered automatically; there's nothing to select via the tag class. Remove the dead attr. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

claude and others added 2 commits March 10, 2026 09:22

fix(py): Implicitly allow pre-releases when version string is a pre-r…

b482dd5

…elease Specifying python_version = "3.15.0a2" now implicitly sets pre_release = True for that major.minor, since the user's intent is unambiguous. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

arrdem force-pushed the arrdem/feat-python-interpreters branch from abae2d9 to b482dd5 Compare March 10, 2026 15:24

arrdem and others added 9 commits March 10, 2026 09:33

test(py): Verify e2e tests use aspect_rules_py interpreter provisioning

badcb9e

Add an e2e test that asserts sys.executable comes from our python_3_* repos, not from rules_python's pythons_hub. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Pre-commit changes

f625fa1

refactor(py): Rename interpreter_version flag to python_version

6bc33d3

Rename //py:interpreter_version to //py:python_version for clarity. Update docs and e2e test .bazelrc references. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Pre-commit changes

66a1db6

docs(uv): Note independence from interpreter provisioning

26cb907

Clarify that uv.project() has no dependency on any particular interpreter provisioning mechanism — it only needs a registered Python toolchain at build time. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

arrdem merged commit b2feee5 into main Mar 10, 2026
3 of 4 checks passed

arrdem deleted the arrdem/feat-python-interpreters branch March 10, 2026 20:52

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(py): Python interpreter provisioning from python-build-standalone#827

feat(py): Python interpreter provisioning from python-build-standalone#827
arrdem merged 36 commits intomainfrom
arrdem/feat-python-interpreters

arrdem commented Mar 6, 2026 •

edited

Loading

Uh oh!

CLAassistant commented Mar 6, 2026 •

edited

Loading

Uh oh!

aspect-workflows Bot commented Mar 6, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

arrdem commented Mar 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Usage

Module scoping

Changes are visible to end-users: yes

Test plan

Uh oh!

CLAassistant commented Mar 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

aspect-workflows Bot commented Mar 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Bazel 8 (Test)

Bazel 9 (Test)

Bazel 8 (Test)

e2e

Bazel 9 (Test)

e2e

Bazel 8 (Test)

examples/uv_pip_compile

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

arrdem commented Mar 6, 2026 •

edited

Loading

CLAassistant commented Mar 6, 2026 •

edited

Loading

aspect-workflows Bot commented Mar 6, 2026 •

edited

Loading