This file explains how to reproduce figures 5, 10, 11, and 12 in our paper.
- Java 7 (the version of Soot that we use does not work with Java 8). We ran our experiments on a machine with Java version "1.7.0_91".
For simplicity, we highly recommend you use the version of our artifact that is pre-installed in this: virtual machine.
- Download link: https://drive.google.com/drive/folders/0B7jf_fJmMJFpQUpXNV9iREMxQTQ
- Username for the VM: user
- Password for the VM: asdf
The dependent-tests-impact folder is under
Run the following steps to set up the Ambari project. Running the following steps takes about an hour. (If you are on the VM, you can skip these steps, because they have already been performed.)
cd dependent-tests-impact/experiments/ambari mvn compile mvn test-compile mvn install -fn -DskipTests dependency:copy-dependencies cd ../ambari-new mvn compile mvn test-compile mvn install -fn -DskipTests dependency:copy-dependencies
mvn install -fn -DskipTests dependency:copy-dependencies step failed for me. See email for details.]]
To reproduce figures 5, 10, 11, and 12 perform the following:
cd dependent-tests-impact/experiments ./figureGenerator.sh
figureGenerator.sh script takes about 14 hours to complete.
The subject program print some error messages to the console. This is expected and normal as some tests do fail and/or produce errors.
Once the script finishes running, the results for prioritization, selection and parallelization can be found in .tex files in directories:
dependent-tests-impact/experiments/ dependent-tests-impact/experiments/prioritization-results/ dependent-tests-impact/experiments/selection-results/ dependent-tests-impact/experiments/parallelization-results/
Since figures 10, 11, and 12 depend on the execution time of the tests,
the results may not be identical to those in the submitted paper.
While the numerical results differ, they still support the
The results used to generate figure 5 in our paper can be found in the
The results used to generate figures 10, 11, and 12 can be found in the
The precomputed dependences can be found in directories
You can re-generate them if you want, but it may take up to 200 hours.
[[Turn the instructions into a script or Maven target, rather than requiring a reader to follow complex, error-prone instructions.]]
To re-generate them:
- delete the files in the directories listed above, and
- uncomment line
prioritization-runner.sh, line 28 in
newExperimentsPrioritizationRunner.sh, line 69 in
selection-runner.sh, line 50 in
newExperimentsSelectionRunner.sh, lines 154 and 156 in
config.sh, and lines 46 and 48 in
However, note that doing so may take 200 hours.
The detailed results for these pre-computed dependences can be found in
The detailed results include the pre-computed dependences that were generated, the time it took to generate such dependency information, etc. These detailed results were also used to generate Figure 9 in our paper.