Add Android emulator testing to workflow

[marcprux]

With #172, we now have the option to build packages for Android from CI. But we don't yet have the ability to run tests.

Performing tests would involve downloading and configuring an Android emulator, ideally customizable with parameters like the Android API level to test. This sort of thing is handled nicely by the venerable android-emulator-runner action, and it is what my swift-android-action uses for testing against Android, but my understanding is that the swiftlang workflows don't want to rely on external actions. So we'd need to manage the setup of the emulator myself. I've put together a sample workflow that does this at https://github.com/swift-android-sdk/swift-android-sdk-workflow-demo/blob/main/.github/workflows/ci.yml to show how this might be accomplished.

The other issue is the complexity of running tests on a separate "host" that doesn't share the same file system. The cross-compiled test binary needs to be transferred over to the QEMU emulator, along with any resources that it depends on. This logic gets hairy, and sometimes requires special parameters to identify special files that need copying over (e.g., SwiftCrypto looks for some test files relative to the test's #filePath path … this won't work without modifying the test case to look elsewhere).

Finally, to support testing both XCTest and Swift Testing tests, there's some fancy footwork required, as discussed at swiftlang/swift-package-manager#8362 (comment). You can see how I do it at swift-android-action's action.yml.

All in all, it is achievable and I think it would be worthwhile for the benefit of actually exercising the tests on an Android emulator rather than just seeing if the source compiles.

[marcprux]

CC: @swiftlang/android-workgroup

[finagolfin]

The sample doesn't look too bad: if you're willing to submit it, I think we should get it in.

Obviously, it won't automatically work with packages that build and use special executables in their tests, like Argument Parser, or that have test data invoked locally through #filePath, but we can add ways to handle those in time, whether through manually listing such required files in the package manifest or this action, or moving package tests away from or reinterpreting #filePath (pinging @jakepetroules and @dschaefer2, who may already be considering such SwiftPM changes or know better workarounds).

I think we should just punt on Testing tests until swiftlang/swift-package-manager#8094 is fixed, as in my experience that makes it impossible to cross-compile many of those Testing repos.

Are these official Android workflows you recently added primarily run in the github runner images or in clean linux Docker containers inside those? Either way, the official github runner images come with most Android tools like the Java SDK and C/C++ NDK pre-installed, so we can always use those easily for our testing.

[marcprux]

Are these official Android workflows you recently added primarily run in the github runner images or in clean linux Docker containers inside those?

Always on the GH runner images directly. I've never tried it in a Docker container, but I think it would work provided the host supported nested virtualization.

[jakepetroules]

or moving package tests away from or reinterpreting #filePath (pinging @jakepetroules and @dschaefer2, who may already be considering such SwiftPM changes or know better workarounds).

The best practice is to use the target's resources property to embed the resources in the test bundle. #filePath should not be used for exactly this reason (cross compilation and remote deployment).