Portable Python.framework Builder for macOS

This project provides a set of shell scripts to build a relocatable (portable) Python.framework for macOS. It downloads Python from Beeware's Python-Apple-support repository, configures it, builds it, slims it down, patches its internal library paths (RPATHs) for portability, compiles bytecode, and signs the entire framework. Optionally, it can include a site-packages directory populated from a requirements.txt file, also with patched binaries.

The goal is to create a Python framework that can be bundled within a macOS application and function correctly without relying on system Python installations or specific install paths.

Features

• Downloads and builds a specific Python version (default: 3.13).
• Supports arm64 and x86_64 architectures (builds for one chosen architecture).
• Modifies Python's release.macOS.exclude to include bin and Python.app.
• Extensively patches dylib IDs and RPATHs of the Python framework and bundled libraries to use @loader_path and @rpath for true portability.
• Slims the framework by removing unnecessary files and directories (tests, docs, etc.).
• Performs architecture-specific slimming (lipo) if not building a universal binary.
• Compiles all .py files to .pyc bytecode.
• Optionally creates a virtual environment using the built framework, installs packages from requirements.txt, and bundles them into the framework's site-packages.
  • Binaries within these site-packages are also RPATH-patched, slimmed, and correctly signed.
• Signs the entire framework and its contents in the correct order using a specified developer identity, handling entitlements appropriately for executables versus libraries.
• Includes smoke tests to verify the final framework's integrity and basic functionality.
• Comprehensive logging for easier debugging.

Prerequisites

1. macOS: These scripts are designed for macOS.
2. Xcode Command Line Tools: Essential for git, make, lipo, codesign, otool, install_name_tool, etc.

   xcode-select --install

3. Homebrew (Recommended): For easily installing an external Python version.

   /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

4. External Python: A Python interpreter matching the major.minor version of PYTHON_VERSION_TAG (see Configuration). This is required for bytecode compilation (compileall). It's best if this is a separate installation (e.g., via Homebrew) and not the system Python.

   # Example for Python 3.13
   brew install python@3.13

5. Apple Developer ID Certificate: For code signing. You need a valid signing identity installed in your keychain.
6. requirements.txt (Optional): If INCLUDE_SITE_PACKAGES is true, this file should be in the project root, listing packages to install.
7. debug.entitlements (Provided): A basic entitlements file allowing debugging is included.

Configuration

Modify these variables at the top of the main build_framework.sh script:

• PYTHON_VERSION_TAG: The tag/branch from Beeware's Python-Apple-support to build (e.g., "3.13", "3.11").
  • Default: "3.13"
• SIGNING_IDENTITY_FILENAME: Name of the file (relative to project root) that will store the chosen signing identity HASH.
  • Default: ".signing_identity"
• ARCHITECTURE: Target architecture. Can be "arm64" or "x86_64". Due to RPATH limitations, universal binaries are not supported by this script.
  • Default: "arm64"
• INCLUDE_SITE_PACKAGES: Set to true or false. If true, packages from requirements.txt will be installed into the framework.
  • Default: true

How to Use

1. Clone the repository (or set up your project with these scripts).
2. Ensure all prerequisites are met.
3. Configure build_framework.sh as described above.
4. Create requirements.txt in the project root if INCLUDE_SITE_PACKAGES=true.
5. Make the main script executable:

   chmod +x build_framework.sh

6. Run the main build script:

   ./build_framework.sh

   The script will prompt you to select a signing identity on the first run (or if the saved one is invalid). Build logs will be saved to the logs/ directory.

Build Process Explained

The build_framework.sh script orchestrates a series of sub-scripts located in the build_framework_scripts/ directory. Each step is logged to the console and to a detailed log file in the logs/ directory.

---

Step 00: Helpers & Initial Setup (00_helpers.sh)

• Explanation: Sets up shared helper functions (logging, environment setup, tool checks) and global settings.
• Technical Details:
  • Defines ANSI color codes for logging.
  • Provides log_info, log_success, log_warning, log_error functions.
  • setup_build_environment: Validates PYTHON_VERSION_TAG and ARCHITECTURE, creates build/ and temp/ directories.
  • check_command_line_tools: Verifies presence of brew, git, make, etc., and finds a compatible external Python for bytecode compilation.
  • liposuction_binary_file: Helper to slim Mach-O universal binaries to a single architecture using lipo -thin.

---

Step 01: Get Python Source (01_get_python.sh)

• Explanation: Downloads or updates the Python source code from Beeware's Python-Apple-support repository for the specified version.
• Technical Details:
  • Clones https://github.com/beeware/Python-Apple-support into temp/Python-Apple-support.
  • If already cloned, fetches latest changes and checks out the tag/branch specified by PYTHON_VERSION_TAG.

---

Step 02: Configure Python Build (02_configure_python.sh)

• Explanation: Modifies the build configuration for the Python source to ensure necessary components like bin (executables) and Python.app are included in the framework.
• Technical Details:
  • Uses sed to comment out lines in temp/Python-Apple-support/patch/Python/release.macOS.exclude that would otherwise exclude Versions/*/bin and Resources/Python.app.

---

Step 03: Build Python.framework (03_build_python_framework.sh)

• Explanation: Compiles the Python source code to produce the initial Python.framework.
• Technical Details:
  • Navigates to temp/Python-Apple-support.
  • Runs make clean and make macOS.
  • Extracts the Python.framework from the resulting tarball (dist/Python-${PYTHON_VERSION_TAG}-macOS-support.custom.tar.gz) into the build/ directory.
  • Cleans up the temp/Python-Apple-support source directory.

---

Step 04: Determine Framework Python Version (04_determine_framework_python_version.sh)

• Explanation: Inspects the built Python.framework to determine the exact Python version (e.g., 3.13.1) and major.minor version (e.g., 3.13). This is needed for subsequent path constructions.
• Technical Details:
  • Reads the symlink build/Python.framework/Versions/Current to find the actual version string.
  • Saves ACTUAL_PYTHON_VERSION and PYTHON_VERSION_MAJOR_MINOR to build/.python_versions_info for use by later scripts.

---

Step 05: Slim Python Framework (05_slim_python.sh)

• Explanation: Removes unnecessary components from the Python.framework to reduce its size.
• Technical Details:
  • Removes directories like lib/pkgconfig, and standard library modules like idlelib, tkinter, test.
  • Removes specific executables from bin/ (e.g., idle3, pydoc3, python3-config).
  • Deletes pre-existing *.pyc, *.pyo files and __pycache__ directories (they will be regenerated later).
  • If ARCHITECTURE is not "universal", uses liposuction_binary_file (and lipo) to slim all Mach-O binaries within the framework to the target architecture.

---

Step 06: Patch Framework Binaries (06_patch_framework_binaries.sh)

• Explanation: This crucial step modifies the internal library linkage paths (RPATHs) of the framework's executables and dynamic libraries to make them portable.
• Technical Details: Uses otool -L to inspect dependencies and install_name_tool to:
  • Change absolute dependency paths in bin/pythonX.Y and Resources/Python.app/Contents/MacOS/Python to relative @loader_path/... references.
  • Set the id of the main Python dylib to the standard @rpath/... format.
  • For other bundled dylibs (e.g., libssl.dylib, libcrypto.dylib):
    • Sets their id to @rpath/<dylib_basename>.
    • Adds @loader_path/. as an RPATH (to find sibling dylibs).
    • Changes their dependencies on other bundled dylibs to use @rpath/....
  • For .so extension modules in lib-dynload/:
    • Cleans up build-time RPATHs.
    • Adds RPATHs (@loader_path/../../ and @loader_path/../../../) to point to the framework's lib/ and Versions/X.Y.Z/ directories.
    • Changes their dependency on the main Python library to @rpath/Python.
    • Changes dependencies on bundled dylibs to @rpath/....

---

Step 07: Compile Python Bytecode (07_compile_python_bytecode.sh)

• Explanation: Compiles all Python source files (.py) within the framework's standard library to bytecode files (.pyc).
• Technical Details:
  • Uses the EXTERNAL_PYTHON_EXEC (found in Step 00) to run python -m compileall -f -q on the framework's lib/pythonX.Y directory.

---

Step 08: Initial Sign Python Framework (08_initial_sign_python.sh)

• Explanation: Performs an initial code signing of all components within the Python.framework. This step allows the framework's Python executable to be used for creating a virtual environment in the next step.
• Technical Details:
  • Prompts the user to select a code signing identity (HASH) if not already saved in SIGNING_IDENTITY_FILENAME.
  • Signs files in a specific order (deepest first):
    • .so files in lib-dynload/ (no entitlements).
    • .dylib files in lib/ (no entitlements).
    • The main Python dylib (no entitlements).
    • Executables in bin/ (with debug.entitlements allowing get-task-allow).
    • Python.app bundle (with debug.entitlements).
    • The top-level Python.framework bundle (no entitlements).
  • Uses codesign --force --sign <identity> --options runtime --timestamp.

---

Step 09: Setup Virtual Environment and Site-Packages (09_setup_venv_and_sitepackages.sh) (Optional)

• Explanation: If INCLUDE_SITE_PACKAGES is true, this step creates a temporary virtual environment using the just-built and signed framework's Python, then installs packages from requirements.txt into it.
• Technical Details:
  • Uses build/Python.framework/Versions/Current/bin/python3 -m venv temp/venv.
  • Upgrades pip in the venv.
  • Installs packages using temp/venv/bin/pip install -r requirements.txt.
  • Moves the site-packages directory from the venv (temp/venv/lib/pythonX.Y/site-packages) to build/PythonPackages for further processing.
  • Cleans up temp/venv.

---

Step 10: Patch Site-Packages Binaries (10_patch_sitepackages_binaries.sh) (Optional)

• Explanation: If site-packages were included, this step patches the RPATHs of any Mach-O binaries (.so, .dylib) within build/PythonPackages to ensure they correctly link against the main Python framework and other libraries.
• Technical Details: Similar to Step 06, but targets binaries in build/PythonPackages.
  • Fixes dylib IDs to @rpath/<basename>.
  • Adds @loader_path/. RPATH to dylibs within packages.
  • Cleans up existing RPATHs.
  • Adds RPATHs pointing to the main framework's library locations (e.g., @loader_path/../../../) and to local library directories within packages (e.g., @loader_path/.dylibs, @loader_path/../lib).
  • Includes special RPATH handling for packages like SciPy and TorchVision.
  • Changes dependencies to link against the main Python framework using its @rpath/... ID and other framework/package dylibs using @rpath/<basename>.

---

Step 11: Slim Site-Packages (11_slim_venv.sh) (Optional)

• Explanation: If site-packages were included, this step removes unnecessary files and directories (tests, docs, metadata) from build/PythonPackages and performs architecture slimming on its binaries.
• Technical Details:
  • Removes common clutter like tests/, docs/, *.egg-info, *.dist-info, __pycache__ from build/PythonPackages.
  • If ARCHITECTURE is not "universal", uses liposuction_binary_file to slim Mach-O binaries in build/PythonPackages.

---

Step 12: Compile Site-Packages Bytecode (12_compile_venv_bytecode.sh) (Optional)

• Explanation: If site-packages were included, this compiles all .py files in build/PythonPackages to .pyc bytecode.
• Technical Details:
  • Uses EXTERNAL_PYTHON_EXEC -m compileall -f -q on build/PythonPackages.

---

Step 13: Move Site-Packages to Framework (13_move_venv_to_framework.sh) (Optional)

• Explanation: If site-packages were processed, this step moves the prepared build/PythonPackages directory into its final location within the Python.framework.
• Technical Details:
  • Moves build/PythonPackages to build/Python.framework/Versions/${ACTUAL_PYTHON_VERSION}/lib/python${PYTHON_VERSION_MAJOR_MINOR}/site-packages.

---

Step 14: Final Sign Framework (14_final_sign_framework.sh)

• Explanation: Performs a final, deep code sign on the entire Python.framework, now including any site-packages. This ensures all new or modified files are correctly signed. A crucial aspect of this step is pre-signing any binaries within site-packages (if included) with appropriate runtime options before the final deep sign of the entire framework.
• Technical Details:
  • If site-packages are included and contain .so or .dylib files, these are first individually signed using codesign --force --sign <identity> --options runtime --timestamp. This ensures they are signed with the hardened runtime but without specific executable entitlements.
  • Then, the entire Python.framework is signed using codesign --force --deep --sign <identity> --options runtime --timestamp --entitlements debug.entitlements Python.framework. The --deep option handles nested components, and the specified entitlements (e.g., debug.entitlements) are applied to the main executables within the framework (like Python.app/Contents/MacOS/Python).
  • Verifies the final signature using codesign --verify --deep --strict.

---

Step 15: Smoke Test Framework (15_smoke_test_framework.sh)

• Explanation: Runs basic tests on the final, signed Python.framework to ensure it's functional.
• Technical Details:
  • Verifies the framework's code signature again.
  • Executes a simple "Hello World" Python script using the framework's Python.app executable.
  • Tests importing a standard library module with C extensions (e.g., ssl).

---

Output

The primary output is a relocatable Python.framework located in the build/ directory. Detailed logs are stored in the logs/ directory.

Troubleshooting

• "ERROR: This script should not be run directly.": Ensure you are running the main build_framework.sh script, not the individual scripts in build_framework_scripts/.
• Missing Command Line Tools: Install Xcode Command Line Tools and any other missing tools reported by 00_helpers.sh.
• Signing Issues:
  • Ensure you have a valid Apple Developer ID certificate in your keychain.
  • If security find-identity shows no identities, you need to set one up.
  • Verify the SIGNING_IDENTITY_FILENAME (default: .signing_identity) contains the correct HASH of your chosen identity.
  • Ensure your main application (if embedding this framework) is signed with the same Team ID as the framework.
• ImportError: dlopen(...so, ...): ... not valid for use in process: mapping process and mapped file (non-platform) have different Team IDs:
  • This is a common code signing error indicating a conflict between the Team ID or entitlements of your app's Python interpreter and the C extension module (.so file) it's trying to load (often from site-packages).
  • The build scripts (specifically 14_final_sign_framework.sh) attempt to mitigate this by:
    1. Pre-signing individual .so/.dylib files in site-packages with the hardened runtime (--options runtime) but without specific executable entitlements.
    2. Performing a --deep sign on the entire Python.framework, applying executable-specific entitlements (like debug.entitlements) only to the framework's main executables.
  • Ensure your Xcode project is embedding and signing the Python.framework correctly and that the Team ID used for signing your main app matches the one used for the framework.
  • Manually verify signatures and entitlements using codesign -dvvv --entitlements - /path/to/binary for your app, the Python executable inside the framework, and the problematic .so file. All TeamIdentifier fields must match.
• Python Version Mismatch for Bytecode Compilation: Ensure the external Python version (e.g., from Homebrew) matches the PYTHON_VERSION_TAG major.minor version.
• RPATH/Linking Errors in Bundled App: If your app using this framework fails with library loading errors (dyld: Library not loaded: ...), use otool -L <your_app_executable> and otool -L <path_to_dylib_in_framework> to investigate. The RPATH patching steps (06 and 10) are critical. Ensure your app's Runpath Search Paths (LD_RUNPATH_SEARCH_PATHS in Xcode) correctly point to the location of the embedded Python.framework (e.g., @executable_path/../Frameworks).
• Check Logs: The logs/build_YYYYMMDD_HHMMSS.log file contains detailed output from all steps.

Contributing

Contributions are welcome! Please feel free to submit pull requests or open issues for bugs, feature requests, or improvements.

License

MIT, please share, improve, help out :D