From 8a21d05fadd1fedb0aeed5f34116bea6b60aabe2 Mon Sep 17 00:00:00 2001 From: Russell Keith-Magee Date: Sun, 30 Jun 2024 08:34:35 +0800 Subject: [PATCH] [3.12] gh-120522: Add a `--with-app-store-compliance` configure option to patch out problematic code (GH-120984) * Add --app-store-compliance configuration option. * Added blurb. * Correct tab-vs-spaces formatting issue. * Correct source file name in docs. Co-authored-by: Nice Zombies * Correct source code reference in Mac docs Co-authored-by: Nice Zombies * Only apply the patch forward, and ensure the working directory is correct. * Make patching reslient to multiple builds. * Documentation fixes found during review Co-authored-by: Alyssa Coghlan * Documentation and configure.ac syntax improvements Co-authored-by: Erlend E. Aasland * Regenerate configure script. * Silence the patch echo output. --------- (cherry picked from commit 48cd104b0cf05dad8958efa9cb9666c029ef9201) Co-authored-by: Russell Keith-Magee Co-authored-by: Nice Zombies Co-authored-by: Alyssa Coghlan Co-authored-by: Erlend E. Aasland --- Doc/library/urllib.parse.rst | 9 ++- Doc/using/configure.rst | 11 ++++ Doc/using/mac.rst | 22 +++++++ Mac/Resources/app-store-compliance.patch | 29 +++++++++ Makefile.pre.in | 17 ++++- ...-06-25-15-29-27.gh-issue-120522.5_n515.rst | 2 + configure | 64 +++++++++++++++++++ configure.ac | 50 +++++++++++++++ 8 files changed, 202 insertions(+), 2 deletions(-) create mode 100644 Mac/Resources/app-store-compliance.patch create mode 100644 Misc/NEWS.d/next/Build/2024-06-25-15-29-27.gh-issue-120522.5_n515.rst diff --git a/Doc/library/urllib.parse.rst b/Doc/library/urllib.parse.rst index 27909b763e9e437..da1089bf8a99e7f 100644 --- a/Doc/library/urllib.parse.rst +++ b/Doc/library/urllib.parse.rst @@ -22,11 +22,18 @@ to an absolute URL given a "base URL." The module has been designed to match the internet RFC on Relative Uniform Resource Locators. It supports the following URL schemes: ``file``, ``ftp``, -``gopher``, ``hdl``, ``http``, ``https``, ``imap``, ``mailto``, ``mms``, +``gopher``, ``hdl``, ``http``, ``https``, ``imap``, ``itms-services``, ``mailto``, ``mms``, ``news``, ``nntp``, ``prospero``, ``rsync``, ``rtsp``, ``rtsps``, ``rtspu``, ``sftp``, ``shttp``, ``sip``, ``sips``, ``snews``, ``svn``, ``svn+ssh``, ``telnet``, ``wais``, ``ws``, ``wss``. +.. impl-detail:: + + The inclusion of the ``itms-services`` URL scheme can prevent an app from + passing Apple's App Store review process for the macOS App Stores. Handling + for the ``itms-services`` scheme *may* be removed if CPython has been built + with the :option:`--with-app-store-compliance` option. + The :mod:`urllib.parse` module defines functions that fall into two broad categories: URL parsing and URL quoting. These are covered in detail in the following sections. diff --git a/Doc/using/configure.rst b/Doc/using/configure.rst index 0e605b38d421d6a..e8d7ed1045c85ec 100644 --- a/Doc/using/configure.rst +++ b/Doc/using/configure.rst @@ -673,6 +673,17 @@ See ``Mac/README.rst``. Specify the name for the python framework on macOS only valid when :option:`--enable-framework` is set (default: ``Python``). +.. option:: --with-app-store-compliance +.. option:: --with-app-store-compliance=PATCH-FILE + + The Python standard library contains strings that are known to trigger + automated inspection tool errors when submitted for distribution by + the macOS App Store. If enabled, this option will apply the list of + patches that are known to correct app store compliance. A custom patch + file can also be specified. This option is disabled by default. + + .. versionadded:: 3.12 + Cross Compiling Options ----------------------- diff --git a/Doc/using/mac.rst b/Doc/using/mac.rst index 31d37aad2a7408e..44fb00de3733c54 100644 --- a/Doc/using/mac.rst +++ b/Doc/using/mac.rst @@ -188,6 +188,28 @@ distributable application: * `PyInstaller `__: A cross-platform packaging tool that creates a single file or folder as a distributable artifact. +App Store Compliance +-------------------- + +Apps submitted for distribution through the macOS App Store must pass Apple's +app review process. This process includes a set of automated validation rules +that inspect the submitted application bundle for problematic code. + +The Python standard library contains some code that is known to violate these +automated rules. While these violations appear to be false positives, Apple's +review rules cannot be challenged. Therefore, it is necessary to modify the +Python standard library for an app to pass App Store review. + +The Python source tree contains +:source:`a patch file ` that will remove +all code that is known to cause issues with the App Store review process. This +patch is applied automatically when CPython is configured with the +:option:`--with-app-store-compliance` option. + +This patch is not normally required to use CPython on a Mac; nor is it required +if you are distributing an app *outside* the macOS App Store. It is *only* +required if you are using the macOS App Store as a distribution channel. + Other Resources =============== diff --git a/Mac/Resources/app-store-compliance.patch b/Mac/Resources/app-store-compliance.patch new file mode 100644 index 000000000000000..f4b7decc01cf1fb --- /dev/null +++ b/Mac/Resources/app-store-compliance.patch @@ -0,0 +1,29 @@ +diff --git a/Lib/test/test_urlparse.py b/Lib/test/test_urlparse.py +index d6c83a75c1c..19ed4e01091 100644 +--- a/Lib/test/test_urlparse.py ++++ b/Lib/test/test_urlparse.py +@@ -237,11 +237,6 @@ def test_roundtrips(self): + '','',''), + ('git+ssh', 'git@github.com','/user/project.git', + '', '')), +- ('itms-services://?action=download-manifest&url=https://example.com/app', +- ('itms-services', '', '', '', +- 'action=download-manifest&url=https://example.com/app', ''), +- ('itms-services', '', '', +- 'action=download-manifest&url=https://example.com/app', '')), + ('+scheme:path/to/file', + ('', '', '+scheme:path/to/file', '', '', ''), + ('', '', '+scheme:path/to/file', '', '')), +diff --git a/Lib/urllib/parse.py b/Lib/urllib/parse.py +index 8f724f907d4..148caf742c9 100644 +--- a/Lib/urllib/parse.py ++++ b/Lib/urllib/parse.py +@@ -59,7 +59,7 @@ + 'imap', 'wais', 'file', 'mms', 'https', 'shttp', + 'snews', 'prospero', 'rtsp', 'rtsps', 'rtspu', 'rsync', + 'svn', 'svn+ssh', 'sftp', 'nfs', 'git', 'git+ssh', +- 'ws', 'wss', 'itms-services'] ++ 'ws', 'wss'] + + uses_params = ['', 'ftp', 'hdl', 'prospero', 'http', 'imap', + 'https', 'shttp', 'rtsp', 'rtsps', 'rtspu', 'sip', diff --git a/Makefile.pre.in b/Makefile.pre.in index 4a957fb0044fcae..71cd8ea40fce77b 100644 --- a/Makefile.pre.in +++ b/Makefile.pre.in @@ -178,6 +178,9 @@ EXPORTSFROM= @EXPORTSFROM@ EXE= @EXEEXT@ BUILDEXE= @BUILDEXEEXT@ +# Name of the patch file to apply for app store compliance +APP_STORE_COMPLIANCE_PATCH=@APP_STORE_COMPLIANCE_PATCH@ + # Short name and location for Mac OS X Python framework UNIVERSALSDK=@UNIVERSALSDK@ PYTHONFRAMEWORK= @PYTHONFRAMEWORK@ @@ -614,7 +617,7 @@ all: @DEF_MAKE_ALL_RULE@ .PHONY: all .PHONY: build_all -build_all: check-clean-src $(BUILDPYTHON) platform sharedmods \ +build_all: check-clean-src @APP_STORE_COMPLIANCE_PATCH_TARGET@ $(BUILDPYTHON) platform sharedmods \ gdbhooks Programs/_testembed scripts checksharedmods rundsymutil .PHONY: build_wasm @@ -848,6 +851,18 @@ SRC_GDB_HOOKS=$(srcdir)/Tools/gdb/libpython.py $(BUILDPYTHON)-gdb.py: $(SRC_GDB_HOOKS) $(INSTALL_DATA) $(SRC_GDB_HOOKS) $(BUILDPYTHON)-gdb.py +# Compliance with app stores (such as iOS and macOS) sometimes requires making +# modifications to the Python standard library. If enabled, apply the patch of +# known modifications to the source tree before building. The patch will be +# applied in a dry-run mode (validating, but not applying the patch) on builds +# that *have* a compliance patch, but where compliance has not been enabled. +build/app-store-compliant: + patch @APP_STORE_COMPLIANCE_PATCH_FLAGS@ --forward --strip=1 --directory="$(srcdir)" --input "$(APP_STORE_COMPLIANCE_PATCH)" + @if test "@APP_STORE_COMPLIANCE_PATCH_FLAGS@" == ""; then \ + mkdir -p build ; \ + echo "$(APP_STORE_COMPLIANCE_PATCH)" > build/app-store-compliant ; \ + fi + # This rule is here for OPENSTEP/Rhapsody/MacOSX. It builds a temporary # minimal framework (not including the Lib directory and such) in the current # directory. diff --git a/Misc/NEWS.d/next/Build/2024-06-25-15-29-27.gh-issue-120522.5_n515.rst b/Misc/NEWS.d/next/Build/2024-06-25-15-29-27.gh-issue-120522.5_n515.rst new file mode 100644 index 000000000000000..4a397a445ccc3b5 --- /dev/null +++ b/Misc/NEWS.d/next/Build/2024-06-25-15-29-27.gh-issue-120522.5_n515.rst @@ -0,0 +1,2 @@ +Added a :option:`--with-app-store-compliance` option to patch out known issues +with macOS/iOS App Store review processes. diff --git a/configure b/configure index d93a4e91f485d7e..b14f22314d7360a 100755 --- a/configure +++ b/configure @@ -978,6 +978,9 @@ HAS_XCRUN EXPORT_MACOSX_DEPLOYMENT_TARGET CONFIGURE_MACOSX_DEPLOYMENT_TARGET _PYTHON_HOST_PLATFORM +APP_STORE_COMPLIANCE_PATCH_FLAGS +APP_STORE_COMPLIANCE_PATCH_TARGET +APP_STORE_COMPLIANCE_PATCH MACHDEP FRAMEWORKINSTALLAPPSPREFIX FRAMEWORKUNIXTOOLSPREFIX @@ -1070,6 +1073,7 @@ enable_universalsdk with_universal_archs with_framework_name enable_framework +with_app_store_compliance with_emscripten_target enable_wasm_dynamic_linking enable_wasm_pthreads @@ -1843,6 +1847,10 @@ Optional Packages: specify the name for the python framework on macOS only valid when --enable-framework is set. see Mac/README.rst (default is 'Python') + --with-app-store-compliance=[PATCH-FILE] + Enable any patches required for compiliance with app + stores. Optional PATCH-FILE specifies the custom + patch to apply. --with-emscripten-target=[browser|node] Emscripten platform --with-suffix=SUFFIX set executable suffix to SUFFIX (default is empty, @@ -4320,6 +4328,62 @@ fi { printf "%s\n" "$as_me:${as_lineno-$LINENO}: result: \"$MACHDEP\"" >&5 printf "%s\n" "\"$MACHDEP\"" >&6; } +{ printf "%s\n" "$as_me:${as_lineno-$LINENO}: checking for --with-app-store-compliance" >&5 +printf %s "checking for --with-app-store-compliance... " >&6; } + +# Check whether --with-app_store_compliance was given. +if test ${with_app_store_compliance+y} +then : + withval=$with_app_store_compliance; + case "$withval" in + yes) + case $ac_sys_system in + Darwin) + APP_STORE_COMPLIANCE_PATCH="Mac/Resources/app-store-compliance.patch" + APP_STORE_COMPLIANCE_PATCH_TARGET="build/app-store-compliant" + APP_STORE_COMPLIANCE_PATCH_FLAGS= + ;; + *) as_fn_error $? "no default app store compliance patch available for $ac_sys_system" "$LINENO" 5 ;; + esac + { printf "%s\n" "$as_me:${as_lineno-$LINENO}: result: applying default app store compliance patch" >&5 +printf "%s\n" "applying default app store compliance patch" >&6; } + ;; + *) + APP_STORE_COMPLIANCE_PATCH="${withval}" + APP_STORE_COMPLIANCE_PATCH_TARGET="build/app-store-compliant" + APP_STORE_COMPLIANCE_PATCH_FLAGS= + { printf "%s\n" "$as_me:${as_lineno-$LINENO}: result: applying custom app store compliance patch" >&5 +printf "%s\n" "applying custom app store compliance patch" >&6; } + ;; + esac + +else $as_nop + + case $ac_sys_system in + Darwin) + # Always *check* the compliance patch on macOS + APP_STORE_COMPLIANCE_PATCH="Mac/Resources/app-store-compliance.patch" + APP_STORE_COMPLIANCE_PATCH_TARGET="build/app-store-compliant" + APP_STORE_COMPLIANCE_PATCH_FLAGS="--dry-run" + { printf "%s\n" "$as_me:${as_lineno-$LINENO}: result: checking (not applying) default app store compliance patch" >&5 +printf "%s\n" "checking (not applying) default app store compliance patch" >&6; } + ;; + *) + # No app compliance patching on any other platform + APP_STORE_COMPLIANCE_PATCH= + APP_STORE_COMPLIANCE_PATCH_TARGET= + APP_STORE_COMPLIANCE_PATCH_FLAGS= + { printf "%s\n" "$as_me:${as_lineno-$LINENO}: result: not patching for app store compliance" >&5 +printf "%s\n" "not patching for app store compliance" >&6; } + ;; + esac + +fi + + + + + if test "$cross_compiling" = yes; then case "$host" in diff --git a/configure.ac b/configure.ac index b46098ae61663cb..337568e0caf313b 100644 --- a/configure.ac +++ b/configure.ac @@ -598,6 +598,56 @@ then fi AC_MSG_RESULT(["$MACHDEP"]) +dnl quadrigraphs "@<:@" and "@:>@" produce "[" and "]" in the output +AC_MSG_CHECKING([for --with-app-store-compliance]) +AC_ARG_WITH( + [app_store_compliance], + [AS_HELP_STRING( + [--with-app-store-compliance=@<:@PATCH-FILE@:>@], + [Enable any patches required for compiliance with app stores. + Optional PATCH-FILE specifies the custom patch to apply.] + )],[ + case "$withval" in + yes) + case $ac_sys_system in + Darwin) + APP_STORE_COMPLIANCE_PATCH="Mac/Resources/app-store-compliance.patch" + APP_STORE_COMPLIANCE_PATCH_TARGET="build/app-store-compliant" + APP_STORE_COMPLIANCE_PATCH_FLAGS= + ;; + *) AC_MSG_ERROR([no default app store compliance patch available for $ac_sys_system]) ;; + esac + AC_MSG_RESULT([applying default app store compliance patch]) + ;; + *) + APP_STORE_COMPLIANCE_PATCH="${withval}" + APP_STORE_COMPLIANCE_PATCH_TARGET="build/app-store-compliant" + APP_STORE_COMPLIANCE_PATCH_FLAGS= + AC_MSG_RESULT([applying custom app store compliance patch]) + ;; + esac + ],[ + case $ac_sys_system in + Darwin) + # Always *check* the compliance patch on macOS + APP_STORE_COMPLIANCE_PATCH="Mac/Resources/app-store-compliance.patch" + APP_STORE_COMPLIANCE_PATCH_TARGET="build/app-store-compliant" + APP_STORE_COMPLIANCE_PATCH_FLAGS="--dry-run" + AC_MSG_RESULT([checking (not applying) default app store compliance patch]) + ;; + *) + # No app compliance patching on any other platform + APP_STORE_COMPLIANCE_PATCH= + APP_STORE_COMPLIANCE_PATCH_TARGET= + APP_STORE_COMPLIANCE_PATCH_FLAGS= + AC_MSG_RESULT([not patching for app store compliance]) + ;; + esac +]) +AC_SUBST([APP_STORE_COMPLIANCE_PATCH]) +AC_SUBST([APP_STORE_COMPLIANCE_PATCH_TARGET]) +AC_SUBST([APP_STORE_COMPLIANCE_PATCH_FLAGS]) + AC_SUBST([_PYTHON_HOST_PLATFORM]) if test "$cross_compiling" = yes; then case "$host" in