Fix sysconfig relocator mangling NDK paths under /usr/local#9
Merged
FeodorFitsner merged 5 commits intoJun 3, 2026
Merged
Conversation
…l prefix
`_install_prefixes` was `(_build_prefix, "/usr/local")`. The second entry
is overly broad and never legitimately matches anything in the Android
cross-compile sysconfig outside of a build-time NDK path that happens to
live under /usr/local (3.13+ CI). When the consumer also has its NDK
under /usr/local — e.g. mobile-forge CI where the runner exports
`NDK_HOME=/usr/local/lib/android/sdk/ndk/<version>` — the substitution
chain breaks:
1. NDK rule rewrites _build_ndk → _local_ndk. _local_ndk starts
with /usr/local on the runner.
2. install-prefix rule rewrites "/usr/local" → _prefix (the consumer's
python install dir). This mangles the path the NDK rule just
correctly resolved.
Result on the mobile-forge runner against the post-flet-dev#8 v3.12 release:
CC = '/home/runner/projects/python-build/android/install/android/
arm64-v8a/python-3.12.12/lib/android/sdk/ndk/<ver>/toolchains/
llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android24-clang'
That path doesn't exist. crossenv reads CC out of sysconfigdata and
exits ENOENT — which is the silent `CalledProcessError` seen in 30+
Android wheel jobs in ndonkoHenri/mobile-forge run 26856516473.
The fix:
- Drop "/usr/local" from `_install_prefixes`. The dedicated
`_build_ndk → _local_ndk` substitution already handles every NDK
path correctly; the install-prefix entry adds no new coverage and
breaks the case where the consumer's NDK happens to share that
prefix.
- Restore the original ordering of the two substitution rules
(install-prefix first, then NDK). flet-dev#8 swapped these as a workaround
for the /usr/local entry's order-dependency. With /usr/local gone
from the tuple, the two rules operate on disjoint substrings and the
order between them is irrelevant — both orders produce identical
output for every consumer environment we ship to (3.12/3.13/3.14 ×
Linux/macOS).
3.12 mobile-forge consumers, post-fix:
CC = '/usr/local/lib/android/sdk/ndk/<ver>/toolchains/llvm/prebuilt/
linux-x86_64/bin/aarch64-linux-android24-clang'
3.13+ macOS dev consumers (the case flet-dev#8 was actually trying to fix),
post-fix:
CC = '~/Library/Android/sdk/ndk/<ver>/toolchains/llvm/prebuilt/
darwin-x86_64/bin/aarch64-linux-android24-clang'
Both correct, no mangling.
Follow-up that should land in a separate PR: add a unit test that
exercises the relocator with NDK_HOME under /usr/local so this exact
regression can't slip in again unnoticed.
A test scaffold under android/tests/ that scales without growing the
workflow file. Tests opt INTO a CI phase via `@pre_build` or
`@post_build` decorators from `_testlib`; the workflow has exactly
two test steps (one per phase) and never has to learn about new
tests as more get added.
Test scoping API (android/tests/_testlib.py):
@pre_build — run before build-all.sh (also runs locally when
MOBILE_FORGE_TEST_PHASE is unset).
@post_build — run after build-all.sh, with the freshly built
install tree accessible via MOBILE_FORGE_INSTALL_TREE.
@requires_python — gate on PYTHON_VERSION_SHORT (one or more values).
All three are stdlib unittest.skipUnless wrappers — no pytest, no
custom test loader. They compose freely (@post_build + @requires_python).
Skip reasons are verbose by design: they tell future-you which env var
was mis-set instead of silently no-running.
Tests landed in this commit:
android/tests/test_normalize_mobile_forge_install.py — @pre_build
Five unit tests for `normalize_mobile_forge_install.append_relocation_block()`.
The first one pins the flet-dev#8 regression that the previous commit
(63f9fe3) fixed: when consumer NDK_HOME contains /usr/local
(mobile-forge runner case), the relocator must not mangle the path
the NDK rule just resolved. Verified to FAIL against pre-fix code
and PASS against the fix:
AssertionError:
'<consumer_prefix>/lib/android/sdk/ndk/<ver>/.../bin/clang' (broken)
vs.
'/.../usr/local/lib/android/sdk/ndk/<ver>/.../bin/clang' (fixed)
android/tests/test_built_install_tree.py — @post_build
Four integration tests that read the freshly built install tree
and assert structural invariants on every shipped sysconfigdata:
- at least one sysconfigdata present (catches build-all.sh
silently producing no artifacts)
- the relocator block marker is present (catches build-all.sh
skipping the normalize_mobile_forge_install.py invocation)
- `_install_prefixes` does NOT contain /usr/local (regression
guard for flet-dev#8 at the shipped artifact level)
- `_build_ndk` is not empty (regression guard for the $toolchain
fallback chain PR flet-dev#8 introduced for 3.13+)
CI wiring (.github/workflows/build-python-version.yml):
- new step before build-all.sh:
`Run android tests (pre-build)`
env: MOBILE_FORGE_TEST_PHASE=pre_build
→ catches source-only regressions in ~1s, before the ~12 min build.
- new step after build-all.sh:
`Run android tests (post-build)`
env: MOBILE_FORGE_TEST_PHASE=post_build,
MOBILE_FORGE_INSTALL_TREE=$GITHUB_WORKSPACE/android/install
→ verifies the shipped artifacts have the structure consumers
(mobile-forge crossenv, flet build) depend on.
Both steps are required to pass — no continue-on-error. Adding new
test files later doesn't touch the workflow.
- Introduced concurrency to cancel in-progress runs on the same ref. - Limited workflow execution to the `main` branch.
There was a problem hiding this comment.
Pull request overview
Fixes a regression in the Android sysconfig relocator where including "/usr/local" in _install_prefixes could corrupt consumer NDK paths (notably on GitHub runners), and adds CI-backed regression coverage to prevent recurrence.
Changes:
- Update the sysconfig relocation block generation to drop
"/usr/local"from_install_prefixes, avoiding NDK path mangling. - Add an
android/tests/unittest scaffold with pre-build unit tests and post-build install-tree validation tests. - Harden GitHub Actions workflows (PR trigger, concurrency cancellation, post-build test step, and gate release publishing to
main).
Reviewed changes
Copilot reviewed 6 out of 7 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
android/normalize_mobile_forge_install.py |
Adjusts relocation substitution rules and improves inline documentation for the injected sysconfig relocator block. |
android/tests/_testlib.py |
Adds phase/version gating decorators for Android test execution in CI. |
android/tests/test_normalize_mobile_forge_install.py |
New unit tests that exercise append_relocation_block() against stub sysconfigdata files to catch relocation regressions. |
android/tests/test_built_install_tree.py |
New post-build tests that validate the shipped install/ tree’s sysconfigdata shape and key baked values. |
.github/workflows/build-python.yml |
Adds PR trigger and workflow-level concurrency settings. |
.github/workflows/build-python-version.yml |
Runs Android pre-/post-build tests and refines job structure/permissions/release gating. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
|
||
| {marker} | ||
| def _mobile_forge_relocate_sysconfig(): | ||
| # Runs once at sysconfigdata import time on the consumer host. Rewrites every path |
…linux_.py The post-build install-tree tests hardcoded the 3.12 sysconfigdata filename (`_sysconfigdata__linux_.py`), which doesn't exist on 3.13+. CPython's official Android tooling (used in the 3.13+ build path) names the file by host triple instead: 3.12 (legacy android-env.sh): _sysconfigdata__linux_.py 3.13+ (CPython's Android tooling): _sysconfigdata__android_<host>-linux-android.py The relocator itself was unaffected — `find_sysconfigdata()` in `normalize_mobile_forge_install.py` already globs `_sysconfigdata__*.py` and applies the relocation block to whatever variant exists. It's purely the post-build tests in `test_built_install_tree.py` that needed the same glob. Symptom in CI: 3.13/3.14 matrix shards failed `test_at_least_one_sysconfigdata_present` (the safety net); the other three post-build tests iterated zero files and degenerated to silent vacuous passes. Fix: switch `_sysconfigdata_files()` to glob `_sysconfigdata__*.py` inside the per-version lib dir (same pattern the production-side `find_sysconfigdata()` uses). Updated the docstring + the `test_at_least_one_sysconfigdata_present` error message to drop the 3.12-only filename mention so future failures point at the right thing. Verified locally by synthesizing a 3.13 install tree with Android-named sysconfigdata files (`_sysconfigdata__android_arm64_v8a-linux-android.py` and `_sysconfigdata__android_x86_64-linux-android.py`) and running the post-build suite — all 4 tests pass.
FeodorFitsner
added a commit
that referenced
this pull request
Jun 3, 2026
`rewrite_build_details_json` hard-coded `/usr/local` as the build-time prefix to substitute. That works for the only consumer schema we ship today (3.14's CPython Android tooling installs to /usr/local), but it violates the narrow-substitution discipline #9 introduced for the sysconfig relocator: an upstream CPython change that moves the install root to anything else would silently miss the rewrite, and unrelated strings under /usr/local that future schema fields might carry would get clobbered. Switch the substitution source to the JSON's own `base_prefix` field. That field is the canonical record of where Python was installed at build time — by construction every other absolute path in the file shares it as a prefix, and reading it from the data instead of from this script's constants means CPython tooling drift gets handled automatically. Idempotent for the same reason as before: once `base_prefix` has been overwritten with the consumer's install path, the build-time string no longer appears anywhere in the file. The early-return on `build_time_prefix == prefix_str` makes that explicit so the function can be safely re-invoked on an already-rewritten tree. Adds android/tests/test_built_build_details_json.py covering: - at least one build-details.json is shipped per ABI (catches the function getting dropped from main() or CPython silently stopping emission of the file) - no `/usr/local` strings survive (narrower than the anchored-at-prefix check below, but targeted at the historical pre-fix shape) - every documented absolute-path field is anchored at the install prefix (catches partial rewrites and build-time-prefix changes) - libpython.dynamic_stableabi points at a file that exists on disk (the linker `-L` target — catches the tarball shipping with the path right but the file missing) Gated `@requires_python("3.14")` because 3.13 and 3.12 don't ship build-details.json (CPython's Android tooling added it in 3.14).
5 tasks
FeodorFitsner
added a commit
that referenced
this pull request
Jun 3, 2026
) * Build Python from source for darwin/iOS; de-pin Linux; bump to 3.12.13/3.13.13/3.14.5 Move iOS/macOS off the beeware Python-Apple-support repo onto CPython's standard build mechanism, and stop hand-pinning the Linux python-build-standalone release. macOS (build_macos.py): universal2 Python.framework built from source for all versions. OpenSSL + xz (+ zstd on 3.14) built universal2 from source (the macOS SDK ships neither OpenSSL nor lzma.h); OpenSSL is shared and bundled into the framework with @rpath install names, matching the released layout; binaries stripped. iOS (build_ios.py): one unified path using CPython's in-tree `Apple build iOS` tool. 3.14 is native; 3.13/3.12 apply a vendored back-port patch (ios_patches/) that adds the Apple tooling (+ the PEP 730 runtime for 3.12) -- no dependency on the Python-Apple-support repo. Reshapes the cross-build output into both the dart bundle and the mobile-forge install/support tree (per-arch dep dirs, VERSIONS, bin stub, platform-config sysconfig), and embeds the iOS deployment version into HOST_GNU_TYPE so mobile-forge's crossenv resolves the iOS release. Linux (resolve_pbs.py): auto-resolve the newest python-build-standalone release for the target micro+arch (PYTHON_DIST_RELEASE kept as optional override); keep x86_64_v2. Workflows: bump matrix to 3.12.13/3.13.13/3.14.5, darwin runner -> macos-26, and only publish GitHub release assets from the main branch. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * Strip dead weight from embedded artifacts (iOS dart ~4x smaller) serious_python loads the stdlib from python-stdlib/ and C extensions from python-xcframeworks/*.fwork, so the full un-pruned stdlib carried inside the iOS Python.xcframework (lib/python3.X, ~200 MB incl. the CPython test suite) and the per-slice lib-dynload/*.so were dead weight in shipped apps. Drop them in package-ios-for-dart.sh: the iOS dart artifact goes from ~89 MB to ~22 MB compressed (308 MB -> 70 MB extracted), with all runtime essentials intact. Also extend the per-platform stdlib excludes (darwin/android/linux/windows) to drop a few more never-needed items: lib2to3 (deprecated; removed upstream in 3.13+), the curses C extensions (the curses package is already excluded), and xxsubtype. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * Strip iOS binaries (consistency with macOS/Android) The Apple build tool leaves the iOS framework binary and extension modules unstripped. Strip local symbols (strip -x, keeps exported PyInit_/C-API symbols needed for dynamic linking) in build_ios.py's reshape, before they become python-xcframeworks. Xcode re-signs on embed. ~4 MB off the extracted iOS dart artifact (70 -> 66 MB). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * Drop interactive/dev-only stdlib modules from embedded packages Running an app in embedded mode never enters the interactive REPL, so the REPL/dev-only modules are dead weight. Confirmed via import trace (a plain script imports none of them; site.py only touches _pyrepl/rlcompleter inside the interactive hook). Remove from all platform excludes: _pyrepl (~300 KB), rlcompleter, tabnanny, the easter eggs this/ antigravity, and the frozen demo modules __hello__/__phello__. Deliberately kept (a library or breakpoint() may import them): unittest (mock), pdb/bdb, pydoc, doctest, code/codeop, multiprocessing, venv. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * Reorganize bundled deps for mobile-forge make_dep_wheels (3.13+) CPython's official Android tooling, used by python-build's 3.13+ build path, builds OpenSSL/bzip2/libffi/xz/SQLite as part of the host build and installs them intermixed under $PREFIX/{include,lib}/. mobile-forge make_dep_wheels.py expects each as a sibling per-lib install dir with its own include/+lib/ — which is what python-build's 3.12 path produces (by downloading separate prebuilt tarballs). For 3.13/3.14 today, the sibling install dirs don't exist and the generated support/<X.Y>/android/VERSIONS file lists no deps, so make_dep_wheels.py emits zero Android dep wheels. Recipes with host requirements like `openssl>=3.0.12` (cryptography) then fail at `pip install --only-binary=:all:` with "No matching distribution found for openssl". Add extract_mobile_forge_deps.py: after the CPython host build copy, detect each bundled lib's version from its header/pkgconfig (or the known-pinned value for bzip2), materialize a sibling install dir at install/android/<abi>/<lib>-<ver>-<N>/{include,lib}/, and append a corresponding `<lib>: <ver>-<N>` entry to VERSIONS. Build number starts at 1 for each dep, independent of the 3.12 counter. Idempotent — re-running replaces sibling dirs and de-duplicates VERSIONS entries so a tree can be re-extracted in place without manual cleanup. After this lands, `source ./setup.sh 3.13.13` will produce Android dep wheels for arm64-v8a + x86_64, and recipes that host-depend on openssl/bzip2/libffi/xz/sqlite resolve normally. * Rewrite paths in build-details.json during normalize CPython 3.14's Android tooling emits `lib/python<X.Y>/build-details.json` that downstream cross-compile tooling (notably `maturin`) reads to decide where libpython lives, where the headers are, and which pkg-config dir to consult. The values are absolute build-time paths rooted at `/usr/local`, so on every consumer machine the file claims libpython sits at `/usr/local/lib/libpython3.14.so` — which is empty. That makes `maturin` add `-L/usr/local/lib` and the linker fails with `unable to find library -lpython3` even when libpython is sitting right there at the install's actual `lib/`. Add a one-shot rewrite to `normalize_mobile_forge_install.py` so the JSON gets re-anchored to the install path at the same time we append the sysconfigdata relocation block. Idempotent (no `/usr/local` remaining after first pass). 3.13 is unaffected because it doesn't ship build-details.json. * Narrow build-details.json rewrite to base_prefix; add post-build tests `rewrite_build_details_json` hard-coded `/usr/local` as the build-time prefix to substitute. That works for the only consumer schema we ship today (3.14's CPython Android tooling installs to /usr/local), but it violates the narrow-substitution discipline #9 introduced for the sysconfig relocator: an upstream CPython change that moves the install root to anything else would silently miss the rewrite, and unrelated strings under /usr/local that future schema fields might carry would get clobbered. Switch the substitution source to the JSON's own `base_prefix` field. That field is the canonical record of where Python was installed at build time — by construction every other absolute path in the file shares it as a prefix, and reading it from the data instead of from this script's constants means CPython tooling drift gets handled automatically. Idempotent for the same reason as before: once `base_prefix` has been overwritten with the consumer's install path, the build-time string no longer appears anywhere in the file. The early-return on `build_time_prefix == prefix_str` makes that explicit so the function can be safely re-invoked on an already-rewritten tree. Adds android/tests/test_built_build_details_json.py covering: - at least one build-details.json is shipped per ABI (catches the function getting dropped from main() or CPython silently stopping emission of the file) - no `/usr/local` strings survive (narrower than the anchored-at-prefix check below, but targeted at the historical pre-fix shape) - every documented absolute-path field is anchored at the install prefix (catches partial rewrites and build-time-prefix changes) - libpython.dynamic_stableabi points at a file that exists on disk (the linker `-L` target — catches the tarball shipping with the path right but the file missing) Gated `@requires_python("3.14")` because 3.13 and 3.12 don't ship build-details.json (CPython's Android tooling added it in 3.14). * Drop duplicate `if:` key on publish-release job Merge of origin/main into darwin-std-pkg landed two copies of the `if: github.ref == 'refs/heads/main'` guard on the publish-release job: one above `needs:` and one below it. YAML mappings can't have duplicate keys, so GitHub Actions rejected the workflow at parse time: Invalid workflow file: .github/workflows/build-python-version.yml#L1 (Line: 204, Col: 5): 'if' is already defined Both copies were the same condition (added independently on each side of the merge), so removing the second one is purely syntactic — no behavior change. --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
#8 introduced a regression in the sysconfig relocator that breaks mobile-forge's Android wheel builds whenever the consumer's
NDK_HOMElives under/usr/local(i.e. every GitHub runner). The_install_prefixes = (_build_prefix, "/usr/local")tuple mangles_local_ndkafter the NDK rule substitutes it in, producing aCCpath like<consumer_prefix>/lib/android/sdk/ndk/.../bin/clangthat doesn't exist —crossenvexits ENOENT, the forge wrapper raises with no captured stderr. See mobile-forge run 26856516473 (30+ Android wheel jobs killed by this).Fix
The
/usr/localentry was redundant (the dedicated_build_ndk → _local_ndkrule already handles every NDK path) and harmful (mangled NDK paths when the consumer's NDK shared the prefix). With it gone, the two substitution rules operate on disjoint substrings and the order between them is irrelevant — restored to the pre-#8 order.Also in this PR
normalize_mobile_forge_install.py.android/tests/scaffold with@pre_build/@post_builddecorators driven by env vars the workflow already exports (stdlibunittest.skipUnless, no pytest dep). 5 unit tests + 4 post-build smoke tests; verified to FAIL against pre-fix code, PASS against this PR.pull_request:trigger,concurrency: cancel-in-progressso PR push-updates don't queue stale ~30 min matrix runs,publish-releasegated ongithub.ref == 'refs/heads/main'.Validation
git checkout origin/main -- normalize_mobile_forge_install.pythen re-run) →test_mobile_forge_runner_ndk_under_usr_localfails with the documented mangling.build-android.gh run rerun --failed, no mobile-forge-side change needed.