ATUA-docker

Introduction

ATUA is a test automation tool for mobile Apps. It focuses on testing methods updated in each software release. The tool aims to maximize the coverage of updated methods and their instructions with a reduced set of inputs. We created a docker for facilitating the experience of ATUA. All you need is build the docker and run a container from built image. This docker contains the ATUA's toolchain (i.e, including AppDiff, Extended-DM2-Instrumenter, Extended-Gator and ATUA testing tool. This docker contains only a case study for demonstration. Some others case studies can be download from here. Due to the limitation of nested virtualization in Windows and Macos, this docker neither supports launching an android emulator. Therefore, the emulator needs to be launched from the host machine.

Instructions

Install Docker

Please follow the installation instruction from Docker's website to install Docker.

Build the ATUA-docker image

There are two ways to build the ATUA-docker image.

1. Build directly from this Git repo.

From the ATUA-docker's directory, run the following command to build the docker image:

docker build -t atua https://github.com/SNTSVV/ATUA-docker.git

• atua is just the name given for the built image.

2. If you want to modify the docker (e.g., change the android platform version), you can clone this git, modify the DockerFile and build the image from the context directory.

From the ATUA-docker's directory, run the following command to build the docker image:

docker build -t atua ./

Run the ATUA-docker image

Since the docker does not contain subject apps except for a case study included for demonstration, we have to mount a shared folder between the container and the host. The shared folder can be used to store the output of the testing. The following command is an example.

docker run -it --rm --mount type=bind,source=/tmp/subjectApps,target=/tmp/data atua bash

This command would start a Bash shell in a new container from the docker image named atua and mount the directory /tmp/subjectApps on the host to the directory /tmp/data in the container. You can also use Volumns for the shared data purpose. Please refer to these documentations for further details:

• docker docs - Use volumes
• docker docs - Use bind mounts

Run ATUA toolset

ATUA toolset is located in /opt/tools/atua-toolset. We also provide an example in this docker image, which is testing the upgrades on the version 134 compared to the version 131 of Activity Diary app.

In this instruction, we demonstrate how to execute the testing tool ATUA since we provide all the necessary inputs for ATUA. You can also follow the manual to reproduce the inputs (i.e., instrumented apk, EWTG) from scratch.

Prepare inputs and outputs directories.

Here, we will create these two directories inside the mounted directory (/tmp/data) and name them inputs and outputs respectively.

mkdir /tmp/data/inputs
mkdir /tmp/data/outputs

Copy the necessary artifacts provided in /opt/tools/atua-toolset/examples

cp -r /opt/tools/atua-toolset/examples/processedArtifacts/de.rampro.activitydiary/131_134/* /tmp/data/inputs

Prepare the configuration for ATUA

In the atua-toolset/template directory, you can find an example configuration file for ATUA named "defaultConfig.properties". Copy it to the input directory created above.

cp /opt/tools/atua-toolset/template/defaultConfig.properties /tmp/data/inputs/myconfig.properties

Change the following lines of myconfig.properties to guide ATUA where the input and output directories are.

Exploration.apksDir=/tmp/data/inputs

Output.outputDir=./tmp/data/outputs

You can change as well the following properties for additional settings.

• The serial number of the emulator Exploration.deviceSerialNumber=emulator-5554

notice that we have already configure the adb host as Core.hostIp=host.docker.internal so that ATUA would search for devices in the host' adb server.

• Time budget for the testing (in minute)

  Selectors.timeLimit=60

Start testing the App with ATUA

When everything is ready, go to the atua directory and execute the following command:

bin/atua --Core-configPath=/tmp/data/inputs/myconfig.properties

When the testing is finished, the outputs are stored in /tmp/data/outputs. The outputs include:

• coverage/: contains details of coverage of each action (generated by DM2)
• report/: contains graph visualization of the entire execution (generated by DM2)
• logcat.log:
• model/de.rampro.activitydiary: contains the model and the coverage related to updated methods.
  • atua-report.txt: contains the overall result in which you can find:
  • The number of statements (instructions)
  • The number of methods
  • The number of updated methods
  • The number of statements belong to the updated methods
  • The coverage
  • Some additional information including:
    • List of covered updated methods
    • List of uncovered update method
    • The coverage ratio after each phase
    • Unreachable windows
  • coverage.txt: contains the list of instructions with the first covered timestamp.
  • crashlist.txt: contains the list of crash exceptions (generated by DM2)
  • EWTG/: contains the enhanced EWTG’s part
  • DSTG/: contains the DSTG generated by ATUA
  • States/: contains the captured GUITrees.
  • trace.csv: contains all the inputs generated and transitions after each action.
  • Images/: contains the screenshots of apps. ATUA takes a screenshot after every input being triggered.
  • actionCoverage.csv: contains the summary of coverage of each action.
  • actionCoverageHTMLReport: contains detail report of each target action (i.e. the actions increase the updated code coverage) under seperated HTML page. The name of an HTML file corresponds to the action’s id that the page reports.