Improve flexibility of Android emulator system image

[freakboy3742]

The Android toolchain currently hard-codes the system image that is used on emulator images built by Briefcase (at time of writing, the default Android-31 image is used). This system image is downloaded when the emulator binary is downloaded, and is never directly updated.

This has some unfortunate side effects:

1. If a user has an emulator created by a different set of tools (e.g., Android Studio), the emulator will appear in the list of available emulators, but won't be able to start
2. A user can't intentionally choose a different base image - e.g., to test older/newer devices.
3. A user can't intentionally choose a base image other than the Default image (e.g., a Google Play enabled image)

Describe the solution you'd like

It would be desirable for Briefcase to:

1. provide the option of selecting a base image from the list provided by the SDK manager
2. check that the system image required by an emulator is available before trying to start the image

Describe alternatives you've considered

There aren't really any alternatives, other than manually invoking sdkmanager to create images and download system images that meet your requirements.

Additional context

sdkmanager is a tool provided by the Android toolchain to do package management. Run briefcase run -v to see examples of how it can be run, including the environment variables that must be set in order to invoke it.

sdkmangager --list will give you a list of available images; any package starting with system_images; is, as the name suggests, a system image. The default system image used by briefcase on M1 hardware is system-images;android-31;default;arm64-v8a (system images are architecture dependent; only architecture-appropriate options should be provided).

sdkmangager --list_installed will give you a list of images that are currently installed.

sdkmanager <package name> will download and install a package.

Android emulators (called AVDs) are stored in ~/.briefcase/avd. There is a .ini config file and a .avd directory for each emulator, with the name of these files matching the emulator name. The .avd folder contains a config.ini that details the system image required by the AVD under the image.sysdir.1 key (e.g., image.sysdir.1=system-images/android-31/default/arm64-v8a/)

[freakboy3742]

For the benefit of anyone interested in this ticket - it should now be significantly easier, as point (2) on the "desired solution" list has been implemented as part of #766. The problem is effectively reduced to:

1. Wrapping sdkmanager --list to generate a list of all available packages
2. Filtering that list to only include system images that correspond to supported Android versions for the current architecture (e.g., no point displaying arm64 images on an x86_64 laptop; no point displaying any image prior to Android-26, as that's the minimum supported version).
3. Displaying the list of candidate images as a choice to the user when creating a new emulator. There is a TODO in the code indicating where this choice should be displayed.

[AndrejusAnto]

Hi, I would like to try to solve this problem. I'm:

1. more than novice in python but not mid and kind of novice in C.
2. i don't like class and OOP, but i could make a "sacrifice" :).
3. i have no experience with android (studio).
   Good enough?

[freakboy3742]

@AndrejusAnto Go right ahead! You don't need any particular experience with Android Studio; the core of the integration with the Android tooling is already in place. The pieces that are needed are almost entirely data parsing, and then converting that parsed data into a set of options for the user. It also shouldn't be extremely complex from a Python point of view.

If I was going to suggest an approach - start by getting familiar with sdkmanager at the command line. How do you invoke it? What output does it generate? Once you're comfortable with sdkmanager, modify the method that is currently invoking sdkmanager so that it captures and parses the output into a Python data structure. Once you've got that data, the last step (showing the choice to the user) should be reasonably straightforward.

[AndrejusAnto]

English is second language and my grammar sucks.
As I understand, it should be something like this at https://github.com/beeware/briefcase/blob/main/src/briefcase/integrations/android_sdk.py#L900
code here https://github.com/AndrejusAnto/Stuff-/blob/master/briefcase.py
For now i am not forking and pull requesting, but if it is too inconvenient, I'll try the right way 😉

[freakboy3742]

Yes, it needs to be a pull request; otherwise it's almost impossible to test, review and revise.

In addition, it's not just a matter of writing the few lines of code that could be copied into place - you need to write tests for that code.

[AndrejusAnto]

Sorry, i forgot to replay. I am not going to fork and PR.

[cryptext]

Hi BeeWare.

A (first ever) pull request was just submitted to try to begin fixing this issue. Hopefully, the problem was understood and what is there is not in the wrong direction. Wanted to make notice to some known issues and reasons for what is there:

• The regular expression that is currently used to filter the system images may not included all the system images you wish to include. Please advise on what is missing, so the regular expression could be updated.
• While trying to figure this out, another issue arose in the verify_system_image() method. While downloading a system image, if the download is incomplete or fails to unpack, due to a bad internet connection, keyboard interrupt, etc., the directories where the system image would be stored could sometimes still be created, but they also could therefore be empty. Then, if the user tries to launch an emulator with that system image again, it will fail, since the original download did not complete. The verify_system_image() method only checked for the existence of the directories and would, therefore, return without trying to download the system image again. A solution seemed to be to try to find if a certain file exists within the expected directory, since that is further along in the download and unpacking process. So, checking for system.img was elected for that reason, but perhaps there is a better way or better file to check? Nonetheless, hope it is proper for this issue to be addressed in this pull request, since it was part of the same process.
• Have never written a Python test before, but willing to learn, so if a test is in fact needed for this pull request, any guidance would be greatly appreciated.

Assuming there will be (many) changes to be made and anticipate your input. Would like to become a regular contributor to the project.

[themartune]

I will take a stab at this.

[themartune]

Currently, the minimum supported version of android is 24. My change will read in the list of available images then filter them based on "system-images", the android version, and the emulator_abi value derived from the host's operating system and architecture.

For example:

Given these emulator choices
ndk;29.0.13113456
sources;android-36
system-images;android-23;google_apis;x86_64
system-images;android-24;android-tv;x86
system-images;android-35;google_atd;arm64-v8a
system-images;android-35;google_apis;x86_64
system-images;android-Baklava;google_apis;x86_64

On a Windows 64-bit machine, we will only return
system-images;android-35;google_apis;x86_64
system-images;android-Baklava;google_apis;x86_64

This may lead to a list that is unmanageably long so we will prompt for the desired android version before showing the available images

[mhsmith]

Currently, the minimum supported version of android is 24

That's correct: it's mentioned in a couple of comments in android_sdk.py. However, we recently made this configurable with the min_os_version setting in pyproject.toml (#2239). The default of 24 is currently coming from briefcase-android-gradle-template/cookiecutter.json, but that's not readily available to Briefcase. So I'd suggest the following approach:

• In android_sdk.py, add a module-level constant for the default minimum version.
• In gradle.py, edit output_format_template_context to set the min_os_version to the default, if it isn't already present. This will override the value in cookiecutter.json when running briefcase create android, making Briefcase the single source of truth for this value.
• When filtering the available images, check the app config for min_os_version, and otherwise use the default. This would require the app object to be passed from run_app to create_emulator.