Join GitHub today
GitHub is home to over 31 million developers working together to host and review code, manage projects, and build software together.Sign up
Execute Additional Test apks #511
For projects that have many modules it's not uncommon to include some small number of android tests in a submodule. Flank is currently unable to run these tests because they are not packaged in the main app's test apk.
In order to run these tests today, a user would need to write a script that passes each library to flank along with the main app's apk for the "app". This is doable, but it would be better if flank could handle this for us.
I'd like to propose the following:
The changes to
Things to consider..
I'd like to work on this, so please let me know if anyone has better ideas on how this might be implemented. I've sort of hacked together the above solution - it's very hacky but it does work and I could clean it up for a PR if it sounds good.
Ah, yea - is that a supported use case though? It seems like you'd end up needing to deal with a lot of chunks of results - one per execution. I was trying to think of a solution that we could build inside Flank to do the same thing as fladle#7 is doing via separate invocations.
For the design, what do you think of this?
flank: additional-apks: - app: ../test_app/apks/app-debug2.apk test: ../test_app/apks/app-debug-androidTest2.apk - test: ../test_app/apks/app-debug-androidTest3.apk
If app is omitted then we'll use the
This allows aggregating multiple runs which I think is a more general solution than only allowing a different test apk.
gcloud supports installing additional APKs but they don't run them.
I'm thinking if we support this, then it should be for an array of app/test pairs. If the app is omitted then we'll use the default app for the provided test apk.
The additional apks from FTL are useful only for allowing you to fill out an inter-app dependency - not for providing additional tests in a secondary apk. The testing APIs offered by FTL via gcloud / rest are targeted at the APK you upload as the main app/test apk. This is one of the topics I discussed with the FTL team.
I think if we want to allow multiple app definitions then we should think about making those a Pair internally so that we can track the main app/test combination. app is a required parameter to run tests in FTL, so we always need to know which one to use even if we make parallel internal executions.
Android libraries package all their logic and tests into a single apk, which means we can use any random app with a library test apk and tests will still execute. That's not the case with an Application module test apk, which requires the matching app.