Skip to content

Commit

Permalink
fix(toolchain): delete 'share/terminfo' for recent linux python toolc…
Browse files Browse the repository at this point in the history
…hains (#1898)

This affects Linux toolchains that have the `terminfo` databases bundled
with the toolchain. Our solution to this is to remove the
`share/terminfo` altogether if we are downloading an affected `linux`
toolchain.

Tested with (on a Mac):
```console
bazel build --platforms=//tests/support:linux_x86_64 @python_3_11//:files
bazel build --platforms=//tests/support:windows_x86_64 @python_3_11//:files
```

Workaround
indygreg/python-build-standalone#231
Fixes #1800
  • Loading branch information
aignas committed May 14, 2024
1 parent 4320d7a commit df55823
Show file tree
Hide file tree
Showing 4 changed files with 73 additions and 5 deletions.
21 changes: 21 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,27 @@ A brief description of the categories of changes:
* Particular sub-systems are identified using parentheses, e.g. `(bzlmod)` or
`(docs)`.

## Unreleased

[x.x.x]: https://github.com/bazelbuild/rules_python/releases/tag/x.x.x

### Changed

### Fixed

### Added

## [0.32.2] - 2024-05-14

[0.32.2]: https://github.com/bazelbuild/rules_python/releases/tag/0.32.2

### Fixed

* Workaround existence of infinite symlink loops on case insensitive filesystems when targeting linux platforms with recent Python toolchains. Works around an upstream [issue][indygreg-231]. Fixes [#1800][rules_python_1800].

[indygreg-231]: https://github.com/indygreg/python-build-standalone/issues/231
[rules_python_1800]: https://github.com/bazelbuild/rules_python/issues/1800

## [0.32.0] - 2024-05-12

[0.32.0]: https://github.com/bazelbuild/rules_python/releases/tag/0.32.0
Expand Down
2 changes: 1 addition & 1 deletion MODULE.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ python.toolchain(
is_default = True,
python_version = "3.11",
)
use_repo(python, "python_versions", "pythons_hub")
use_repo(python, "python_3_11", "python_versions", "pythons_hub")

# This call registers the Python toolchains.
register_toolchains("@pythons_hub//:all")
Expand Down
29 changes: 25 additions & 4 deletions python/repositories.bzl
Original file line number Diff line number Diff line change
Expand Up @@ -176,7 +176,7 @@ def _python_repository_impl(rctx):
rctx.patch(patch, strip = 1)

# Write distutils.cfg to the Python installation.
if "windows" in rctx.os.name:
if "windows" in platform:
distutils_path = "Lib/distutils/distutils.cfg"
else:
distutils_path = "lib/python{}/distutils/distutils.cfg".format(python_short_version)
Expand All @@ -187,7 +187,7 @@ def _python_repository_impl(rctx):

# Make the Python installation read-only.
if not rctx.attr.ignore_root_user_error:
if "windows" not in rctx.os.name:
if "windows" not in platform:
lib_dir = "lib" if "windows" not in platform else "Lib"

repo_utils.execute_checked(
Expand Down Expand Up @@ -228,7 +228,28 @@ def _python_repository_impl(rctx):
"**/__pycache__/*.pyc.*", # During pyc creation, temp files named *.pyc.NNN are created
]

if rctx.attr.ignore_root_user_error or "windows" in rctx.os.name:
if "linux" in platform:
# Workaround around https://github.com/indygreg/python-build-standalone/issues/231
for url in urls:
head_and_release, _, _ = url.rpartition("/")
_, _, release = head_and_release.rpartition("/")
if not release.isdigit():
# Maybe this is some custom toolchain, so skip this
break

if int(release) >= 20240224:
# Starting with this release the Linux toolchains have infinite symlink loop
# on host platforms that are not Linux. Delete the files no
# matter the host platform so that the cross-built artifacts
# are the same irrespective of the host platform we are
# building on.
#
# Link to the first affected release:
# https://github.com/indygreg/python-build-standalone/releases/tag/20240224
rctx.delete("share/terminfo")
break

if rctx.attr.ignore_root_user_error or "windows" in platform:
glob_exclude += [
# These pycache files are created on first use of the associated python files.
# Exclude them from the glob because otherwise between the first time and second time a python toolchain is used,"
Expand Down Expand Up @@ -263,7 +284,7 @@ def _python_repository_impl(rctx):
]

if rctx.attr.coverage_tool:
if "windows" in rctx.os.name:
if "windows" in platform:
coverage_tool = None
else:
coverage_tool = '"{}"'.format(rctx.attr.coverage_tool)
Expand Down
26 changes: 26 additions & 0 deletions tests/support/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -37,3 +37,29 @@ platform(
"@platforms//os:windows",
],
)

# Used when testing downloading of toolchains for a different platform

platform(
name = "linux_x86_64",
constraint_values = [
"@platforms//cpu:x86_64",
"@platforms//os:linux",
],
)

platform(
name = "mac_x86_64",
constraint_values = [
"@platforms//cpu:x86_64",
"@platforms//os:macos",
],
)

platform(
name = "windows_x86_64",
constraint_values = [
"@platforms//cpu:x86_64",
"@platforms//os:windows",
],
)

0 comments on commit df55823

Please sign in to comment.