Mechanism of the solution

Overview

To execute a run on Firebase test lab you can either use their web application or CLI (command line interface). We use the CLI because we run the tests through our continuous integration servers.

In the command you specify many flags like:

• --device The device model and API.
• --timeout The max time this test execution can run before it is cancelled.
• --test-targets The tests to run. You can target a certain package, certain class, or even a certain method to run. Check a subset of the different values that can be substituted for the flag:

--test-targets "package com.my.package.name"            # Certain package 
--test-targets "class com.foo.ClassName"                # Certain class
--test-targets "class com.foo.ClassName#testMethodName" # Certain method

Annotation processor:

Our implemented annotation processor scans the whole code base and prints the names of all UI test cases, at the build time, in a text file ui-tests in the project root directory. Check this example of a printed file for a virtual code base:

com.package.testClass1#testMethodA
com.package.testClass2#testMethodB
com.package.testClass3#testMethodC
com.package.testClass4#testMethodD
com.package.testClass5#testMethodE

Parallel execution of shell scripts:

A shell script which uses the file containing tests, outputted by the annotation processor, as an input:

• Prepares the tests names to match the specified format of the command.
• Finds the number of available parallel jobs on the machine.
• Divides the number of tests by the number of jobs.
• Triggers parallel runs to Firebase
• Blocks until all the runs returns their results.
• Outputs the overall result of all tests.