Skip to content
Automatic Repair Framework that abstract repair tools and bug benchmarks
Branch: master
Clone or download
Type Name Latest commit message Commit time
Failed to load latest commit information.
benchmarks update defects4j Feb 16, 2019
data Clean files Jun 10, 2019
libs remove useless files Nov 28, 2018
repair_tools Clean files Jun 10, 2019
script normalize Arja output result Jun 10, 2019
.gitignore first version of bears Oct 29, 2018
.gitmodules use https instead of ssh in gitmodules Jun 10, 2019
Dockerfile update Dockerfile Jun 10, 2019 put the default configuration after the init Jun 11, 2019
LICENSE Update and add complementary files Jun 9, 2019 Update and Jun 10, 2019 improve file Jun 10, 2019

The RepairThemAll Framework

RepairThemAll is a framework that allows the execution of automatic program repair tools on benchmarks of bugs by providing an abstraction around the repair tools and the benchmarks.

If you use RepairThemAll, please cite our paper:

  author    = {Thomas Durieux and Fernanda Madeiral and Matias Martinez and Rui Abreu},
  title     = {{Empirical Review of Java Program Repair Tools: A Large-Scale Experiment on 2,141 Bugs and 23,551 Repair Attempts}},
  booktitle = {Proceedings of the 27th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE '19)},
  year      = {2019},
  url       = {}

Table of Contents

  1. Features
  2. Supported repair tools
  3. Supported benchmarks of bugs
  4. Repository structure
  5. Usage
  6. Extending RepairThemAll to add support for different repair tools and benchmarks of bugs
  7. Reproduce the experiment reported in the paper using RepairThemAll

1. Features

  • Execution of repair tools on benchmarks of bugs
  • Configuration of the environment to execute repair tools properly on the bugs
  • Multi-process execution
  • Grid5000 cluster support

2. Supported repair tools

# Tool Language Repository Commit id
1 Nopol Java 7ba58a78d
2 DynaMoth Java 7ba58a78d
3 NPEFix Java 403445b9a
4 jGenProg Java 26ee3dfc8
5 jKali Java 26ee3dfc8
6 jMutRepair Java 26ee3dfc8
7 Cardumen Java 26ee3dfc8
8 ARJA Java e60b990f9
9 GenProg-A Java e60b990f9
10 RSRepair-A Java e60b990f9
11 Kali-A Java e60b990f9

3. Supported benchmarks of bugs

# Benchmark Language # Projects # Bugs Link
1 Bears Java 71 251
2 Bugs.jar Java 8 1,158
3 Defects4J Java 6 395
4 IntroClassJava Java 6 297
5 QuixBugs Java 40 40
Total 130 2,051

4. Repository structure

This repository is structured as follow:

├── benchmarks: contains a git submodule per benchmark plugged-in RepairThemAll
├── data: 
│ ├── benchmarks: contains additional information/files on benchmarks
│ ├── repair_tools: contains information on repair tools, e.g. the launcher class name
├── libs: contains the dependencies
├── repair_tools: contains a jar file per repair tool (e.g. npefix.jar) or per repair framework where several repair tools are implemented (e.g. astor.jar)
├── script: contains the actual scripts of RepairThemAll that use everything mentioned above to run repair tools on benchmarks of bugs
│ ├── Main files:
│ ├── contains the parameters of RepairThemAll (see file)
│ ├── contains the script to run a repair tool on a benchmark
│ └── contains the script to print the bugs available from a benchmark

5. Usage

RepairThemAll can be executed directly from the source or via a Docker image where RepairThemAll has been pre-configured and is ready to be used. One can find the usage instructions here.

6. Extending RepairThemAll to add support for different repair tools and benchmarks of bugs

To add a new benchmark or a new repair tool, see the instructions here.

7. Reproduce the experiment reported in the paper using RepairThemAll

The results reported in the paper are at program-repair/RepairThemAll_experiment.

In order to reproduce those results, the 11 repair tools need to be executed on the 5 benchmarks. Warning: the experiment took 313 days of combined execution time.

That execution requires a command line for each pair of repair tool and benchmark (totaling 55 command lines) as following:

python script/ Nopol --benchmark Defects4J
python script/ Nopol --benchmark Bears
You can’t perform that action at this time.