Merge upstream (#2)

* Add support --res_xml option in p4a * res_xml paths are relative to buildozer.spec * Changes for NDK23 (kivy#1427) The arch no longer used in url for NDK23+ * Our self-hosted Apple Silicon runner now has been migrated to actions/runner v2.292.0 which now supports arm64 natively (kivy#1438) * use p4a --add-source instead of manual copy (kivy#1450) Currently, android.add_src does not work anymore. Using --add-source from p4a make it work again. * fix aar build (kivy#1444) * fix aar build * update default.spec to include 'debug_artifact' * Updates default buildozer.spec NDK from 19b to 23b (kivy#1462) * Update CHANGELOG and prepare release 1.4.0 (kivy#1463) * Bump version to 1.4.1.dev0 (kivy#1464) * p4a prerequisites install should be done in non-interactive mode during CI builds. (kivy#1465) * Avoid the usage of `shell=True` in Popen, which could lead to potential security risks. (kivy#1435) * Removes (where possible) the usage of shell=True in Popen * Fixes (needs check on runtime) adb + add a solution for p4a.extra_args * Change the logic to auto accept the SDK licenses, to avoid the shell=True usage * Removes six dependency in tests (kivy#1475) * Fixes some E275 - assert is a keyword. (kivy#1495) * Fix presplash color (kivy#1497) Buildozer wont build if a presplash color is set, this should fix it as descrived in kivy#1487 * Show output during aab support check, as p4a may require the user input (kivy#1494) * Update installation.rst (kivy#1500) * Update installation.rst Update jdk and Ubuntu versions. Update WSL usage instructions. * Update installation.rst * Update docs/source/installation.rst Co-authored-by: Mirko Galimberti <me@mirkogalimberti.com> * Update docs/source/installation.rst Co-authored-by: Mirko Galimberti <me@mirkogalimberti.com> Co-authored-by: Mirko Galimberti <me@mirkogalimberti.com> Co-authored-by: Eero af Heurlin <eero.afheurlin@iki.fi> Co-authored-by: HyTurtle <81598434+HyTurtle@users.noreply.github.com> Co-authored-by: Mirko Galimberti <me@mirkogalimberti.com> Co-authored-by: Mathieu Virbel <mat@meltingrocks.com> Co-authored-by: Mikhail Zakharov <mzakharo@users.noreply.github.com> Co-authored-by: Akshay Arora <akshayaurora@gmail.com> Co-authored-by: Apacelus <46828271+apacelus@users.noreply.github.com> Co-authored-by: RobertF <34464649+RobertFlatt@users.noreply.github.com>
Craulback · Sep 27, 2022 · a58ff06 · a58ff06
1 parent fbf9371
commit a58ff06
Show file tree

Hide file tree

Showing 12 changed files with 374 additions and 277 deletions.
diff --git a/.github/workflows/android.yml b/.github/workflows/android.yml
@@ -1,4 +1,8 @@
 on: [push, pull_request]
+
+env:
+  PYTHONFORANDROID_PREREQUISITES_INSTALL_INTERACTIVE: 0
+
 name: Android
 jobs:
   Integration:

diff --git a/.github/workflows/ios.yml b/.github/workflows/ios.yml
@@ -3,17 +3,13 @@ name: iOS
 jobs:
   Integration:
     name: "Integration (${{ matrix.runs_on }}, ${{ matrix.python }})"
-    defaults:
-      run:
-        shell: ${{ matrix.run_wrapper || 'bash --noprofile --norc -eo pipefail {0}' }}
     runs-on: ${{ matrix.runs_on || 'macos-latest' }}
     strategy:
       matrix:
         include:
           - runs_on: macos-latest
             python: '3.9'
           - runs_on: apple-silicon-m1
-            run_wrapper: arch -arm64 bash --noprofile --norc -eo pipefail {0}
             python: '3.9.7'
     steps:
     - name: Setup python

diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -1,5 +1,58 @@
 # Change Log
 
+## [1.4.0](https://github.com/kivy/buildozer/tree/1.4.0) (2022-07-20)
+
+[Full Changelog](https://github.com/kivy/buildozer/compare/1.3.0...1.4.0)
+
+**Closed issues:**
+
+- error [\#1461](https://github.com/kivy/buildozer/issues/1461)
+- error [\#1460](https://github.com/kivy/buildozer/issues/1460)
+- ModuleNotFoundError: No module named '\_bz2' [\#1457](https://github.com/kivy/buildozer/issues/1457)
+- `java.nio.file.NoSuchFileException` [\#1456](https://github.com/kivy/buildozer/issues/1456)
+- kivy app doesnt start on android [\#1455](https://github.com/kivy/buildozer/issues/1455)
+- \(WSL\) buildozer adb doesn't recognize usb [\#1453](https://github.com/kivy/buildozer/issues/1453)
+- Error: Command \[...\] returned non-zero exit status 1 [\#1452](https://github.com/kivy/buildozer/issues/1452)
+- Command failed: /home/username/.buildozer/android/platform/android-sdk/tools/bin/sdkmanager --sdk\_root=/home/username/.buildozer/android/platform/android-sdk platform-tools [\#1449](https://github.com/kivy/buildozer/issues/1449)
+- gradlew assembly Debug execution error when building apk [\#1447](https://github.com/kivy/buildozer/issues/1447)
+- How to run Python script in background in android? [\#1446](https://github.com/kivy/buildozer/issues/1446)
+- buildozer appclean doesnt work. [\#1443](https://github.com/kivy/buildozer/issues/1443)
+- Flags android.gradle\_dependencies && android.add\_jars don't work. [\#1442](https://github.com/kivy/buildozer/issues/1442)
+- ModuleNotFoundError: No module named 'PIL' [\#1440](https://github.com/kivy/buildozer/issues/1440)
+- Bulldozer error while running "buildozer -v android debug" [\#1439](https://github.com/kivy/buildozer/issues/1439)
+- started app on three different andoid phones - there is no internet connection on 2 of them [\#1434](https://github.com/kivy/buildozer/issues/1434)
+- Not Able to change the python version under buildozer 1.3.0 [\#1432](https://github.com/kivy/buildozer/issues/1432)
+- ndk r22 invalid  [\#1431](https://github.com/kivy/buildozer/issues/1431)
+- error building numpy with bulldozer [\#1426](https://github.com/kivy/buildozer/issues/1426)
+- sh.CommandNotFound: cmake [\#1424](https://github.com/kivy/buildozer/issues/1424)
+- Compile-time name 'JNIUS\_PYTHON3' not defined [\#1422](https://github.com/kivy/buildozer/issues/1422)
+- Error Building hostpython3 for arm64-v8a with buildozer [\#1421](https://github.com/kivy/buildozer/issues/1421)
+- building for android failed [\#1420](https://github.com/kivy/buildozer/issues/1420)
+- Error "Aidl not found, please install it." [\#1416](https://github.com/kivy/buildozer/issues/1416)
+- Buildozer Release Signign Issue [\#1415](https://github.com/kivy/buildozer/issues/1415)
+- Unable to run the buildozer command on initial application [\#1414](https://github.com/kivy/buildozer/issues/1414)
+- buildozer -v android debug error,please help me,thank you! [\#1413](https://github.com/kivy/buildozer/issues/1413)
+- Buildozer failed to create android app [\#1412](https://github.com/kivy/buildozer/issues/1412)
+- module encodings found error or something else [\#1408](https://github.com/kivy/buildozer/issues/1408)
+- \[ERROR\]:   Build failed: Requested API target 27 is not available, install it with the SDK android tool. [\#1404](https://github.com/kivy/buildozer/issues/1404)
+- buildozer requirements for firebase-admin Python?  [\#1402](https://github.com/kivy/buildozer/issues/1402)
+- I am given the following error, which I believe is due to recent changes for aab support in buildozer:  " This buildozer version requires a python-for-android version with AAB \(Android App Bundle\) support. Please update your pinned version accordingly [\#1401](https://github.com/kivy/buildozer/issues/1401)
+- How to mention Python modules used in Kivy buildozer.spec file? [\#1400](https://github.com/kivy/buildozer/issues/1400)
+- Приложение вылетает при запуске [\#1399](https://github.com/kivy/buildozer/issues/1399)
+- Buildozer deploy failed with python3 -m venv venv command [\#1389](https://github.com/kivy/buildozer/issues/1389)
+- Gradlew Build error for Android on Linux [\#1371](https://github.com/kivy/buildozer/issues/1371)
+- AAB [\#1353](https://github.com/kivy/buildozer/issues/1353)
+- Error creating apk on Macos Big Sur [\#1345](https://github.com/kivy/buildozer/issues/1345)
+- BUILD FAILED [\#1335](https://github.com/kivy/buildozer/issues/1335)
+
+**Merged pull requests:**
+
+- Updates default buildozer.spec NDK from 19b to 23b [\#1462](https://github.com/kivy/buildozer/pull/1462) ([misl6](https://github.com/misl6))
+- use p4a --add-source instead of manual copy of java files [\#1450](https://github.com/kivy/buildozer/pull/1450) ([tito](https://github.com/tito))
+- fix aar build [\#1444](https://github.com/kivy/buildozer/pull/1444) ([mzakharo](https://github.com/mzakharo))
+- Our self-hosted Apple Silicon runner now has been migrated to actions/runner v2.292.0 which now supports arm64 natively [\#1438](https://github.com/kivy/buildozer/pull/1438) ([misl6](https://github.com/misl6))
+- Changes for NDK23 [\#1427](https://github.com/kivy/buildozer/pull/1427) ([HyTurtle](https://github.com/HyTurtle))
+- Bump version to 1.4.0.dev0 [\#1411](https://github.com/kivy/buildozer/pull/1411) ([misl6](https://github.com/misl6))
 
 ## [1.3.0](https://github.com/kivy/buildozer/tree/1.3.0) (2022-03-13)
 [Full Changelog](https://github.com/kivy/buildozer/compare/1.2.0...1.3.0)

diff --git a/buildozer/__init__.py b/buildozer/__init__.py
@@ -6,7 +6,7 @@
 
 '''
 
-__version__ = '1.4.0.dev0'
+__version__ = '1.4.1.dev0'
 
 import os
 import re
@@ -26,6 +26,8 @@
 from fnmatch import fnmatch
 
 from pprint import pformat
+import shlex
+import pexpect
 
 from urllib.request import FancyURLopener
 from configparser import ConfigParser
@@ -157,7 +159,7 @@ def set_target(self, target):
     def prepare_for_build(self):
         '''Prepare the build.
         '''
-        assert(self.target is not None)
+        assert self.target is not None
         if hasattr(self.target, '_build_prepared'):
             return
 
@@ -188,8 +190,8 @@ def build(self):
 
         (:meth:`prepare_for_build` must have been call before.)
         '''
-        assert(self.target is not None)
-        assert(hasattr(self.target, '_build_prepared'))
+        assert self.target is not None
+        assert hasattr(self.target, '_build_prepared')
 
         if hasattr(self.target, '_build_done'):
             return
@@ -261,7 +263,6 @@ def cmd(self, command, **kwargs):
         kwargs.setdefault('stdout', PIPE)
         kwargs.setdefault('stderr', PIPE)
         kwargs.setdefault('close_fds', True)
-        kwargs.setdefault('shell', True)
         kwargs.setdefault('show_output', self.log_level > 1)
 
         show_output = kwargs.pop('show_output')
@@ -357,7 +358,6 @@ def cmd(self, command, **kwargs):
                 process.returncode)
 
     def cmd_expect(self, command, **kwargs):
-        from pexpect import spawnu
 
         # prepare the environ, based on the system + our own env
         env = environ.copy()
@@ -378,7 +378,7 @@ def cmd_expect(self, command, **kwargs):
             self.debug('Run (expect) {0!r} ...'.format(command.split()[0]))
 
         self.debug('Cwd {}'.format(kwargs.get('cwd')))
-        return spawnu(command, **kwargs)
+        return pexpect.spawnu(shlex.join(command), **kwargs)
 
     def check_configuration_tokens(self):
         '''Ensure the spec file is 'correct'.
@@ -515,9 +515,11 @@ def check_application_requirements(self):
     def _install_application_requirement(self, module):
         self._ensure_virtualenv()
         self.debug('Install requirement {} in virtualenv'.format(module))
-        self.cmd('pip install --target={} {}'.format(self.applibs_dir, module),
-                 env=self.env_venv,
-                 cwd=self.buildozer_dir)
+        self.cmd(
+            ["pip", "install", f"--target={self.applibs_dir}", module],
+            env=self.env_venv,
+            cwd=self.buildozer_dir,
+        )
 
     def check_garden_requirements(self):
         garden_requirements = self.config.getlist('app',
@@ -530,13 +532,15 @@ def _ensure_virtualenv(self):
             return
         self.venv = join(self.buildozer_dir, 'venv')
         if not self.file_exists(self.venv):
-            self.cmd('python3 -m venv ./venv',
+            self.cmd(["python3", "-m", "venv", "./venv"],
                     cwd=self.buildozer_dir)
 
         # read virtualenv output and parse it
-        output = self.cmd('bash -c "source venv/bin/activate && env"',
-                get_stdout=True,
-                cwd=self.buildozer_dir)
+        output = self.cmd(
+            ["bash", "-c", "source venv/bin/activate && env"],
+            get_stdout=True,
+            cwd=self.buildozer_dir,
+        )
         self.env_venv = copy(self.environ)
         for line in output[0].splitlines():
             args = line.split('=', 1)
@@ -594,22 +598,22 @@ def file_copy(self, source, target, cwd=None):
 
     def file_extract(self, archive, cwd=None):
         if archive.endswith('.tgz') or archive.endswith('.tar.gz'):
-            self.cmd('tar xzf {0}'.format(archive), cwd=cwd)
+            self.cmd(["tar", "xzf", archive], cwd=cwd)
             return
 
         if archive.endswith('.tbz2') or archive.endswith('.tar.bz2'):
             # XXX same as before
-            self.cmd('tar xjf {0}'.format(archive), cwd=cwd)
+            self.cmd(["tar", "xjf", archive], cwd=cwd)
             return
 
         if archive.endswith('.bin'):
             # To process the bin files for linux and darwin systems
-            self.cmd('chmod a+x {0}'.format(archive), cwd=cwd)
-            self.cmd('./{0}'.format(archive), cwd=cwd)
+            self.cmd(["chmod", "a+x", archive], cwd=cwd)
+            self.cmd([f"./{archive}"], cwd=cwd)
             return
 
         if archive.endswith('.zip'):
-            self.cmd('unzip -q {}'.format(join(cwd, archive)), cwd=cwd)
+            self.cmd(["unzip", "-q", join(cwd, archive)], cwd=cwd)
             return
 
         raise Exception('Unhandled extraction for type {0}'.format(archive))

diff --git a/buildozer/default.spec b/buildozer/default.spec
@@ -108,7 +108,7 @@ fullscreen = 0
 #android.sdk = 20
 
 # (str) Android NDK version to use
-#android.ndk = 19b
+#android.ndk = 23b
 
 # (int) Android NDK API to use. This is the minimum API your app will support, it should usually match android.minapi.
 #android.ndk_api = 21
@@ -228,6 +228,9 @@ fullscreen = 0
 # (str) XML file to include as an intent filters in <activity> tag
 #android.manifest.intent_filters =
 
+# (list) Copy these files to src/main/res/xml/ (used for example with intent-filters)
+#android.res_xml = PATH_TO_FILE,
+
 # (str) launchMode to set for the main activity
 #android.manifest.launch_mode = standard
 
@@ -287,9 +290,12 @@ android.allow_backup = True
 # (bool) disables the compilation of py to pyc/pyo files when packaging
 # android.no-compile-pyo = True
 
-# (str) The format used to package the app for release mode (aab or apk).
+# (str) The format used to package the app for release mode (aab or apk or aar).
 # android.release_artifact = aab
 
+# (str) The format used to package the app for debug mode (apk or aar).
+# android.debug_artifact = apk
+
 #
 # Python for android (p4a) specific
 #
@@ -332,6 +338,7 @@ android.allow_backup = True
 #p4a.extra_args =
 
 
+
 #
 # iOS specific
 #

diff --git a/buildozer/target.py b/buildozer/target.py
@@ -102,7 +102,7 @@ def cmd_update(self, *args):
     def cmd_debug(self, *args):
         self.buildozer.prepare_for_build()
         self.build_mode = 'debug'
-        self.artifact_format = 'apk'
+        self.artifact_format = self.buildozer.config.getdefault('app', 'android.debug_artifact', 'apk')
         self.buildozer.build()
 
     def cmd_release(self, *args):
@@ -228,8 +228,6 @@ def path_or_git_url(self, repo, owner='kivy', branch='master',
             branch = config.getdefault('app', '{}_branch'.format(key), branch)
             default_url = url_format.format(owner=owner, repo=repo, branch=branch)
             url = config.getdefault('app', '{}_url'.format(key), default_url)
-            if branch != 'master':
-                url = "--branch {} {}".format(branch, url)
         return path, url, branch
 
     def install_or_update_repo(self, repo, **kwargs):
@@ -252,15 +250,14 @@ def install_or_update_repo(self, repo, **kwargs):
         custom_dir, clone_url, clone_branch = self.path_or_git_url(repo, **kwargs)
         if not self.buildozer.file_exists(install_dir):
             if custom_dir:
-                cmd('mkdir -p "{}"'.format(install_dir))
-                cmd('cp -a "{}"/* "{}"/'.format(custom_dir, install_dir))
+                cmd(["mkdir", "-p", install_dir])
+                cmd(["cp", "-a", f"{custom_dir}/*", f"{install_dir}/"])
             else:
-                cmd('git clone {}'.format(clone_url),
-                        cwd=self.buildozer.platform_dir)
+                cmd(["git", "clone", "--branch", clone_branch, clone_url], cwd=self.buildozer.platform_dir)
         elif self.platform_update:
             if custom_dir:
-                cmd('cp -a "{}"/* "{}"/'.format(custom_dir, install_dir))
+                cmd(["cp", "-a", f"{custom_dir}/*", f"{install_dir}/"])
             else:
-                cmd('git clean -dxf', cwd=install_dir)
-                cmd('git pull origin {}'.format(clone_branch), cwd=install_dir)
+                cmd(["git", "clean", "-dxf"], cwd=install_dir)
+                cmd(["git", "pull", "origin", clone_branch], cwd=install_dir)
         return install_dir