This github page contains the results for our paper on program repair submitted at FSE 2017.
Our results are measured by the number of correct patches generated by each repair tool, and the overfitting rate which is the number of incorrect repairs divided by the total number of produced repairs of each tool.
We show that on the dataset of 100 large programs, the overfitting rate of Angelix, Enumerative, and CVC4 are 54%, 54%, and 58%, as shown in the Table 4 in our paper.
The data in this page is structured as follows.
====Results on the large program dataset, in which our tool can correctly fix 20 programs========
Each of the following folders show the patches generated by each repair tool: CVC4, S3, Enum, and angelix for the 20 programs that S3 can correctly fix.
-prioritizeCVC4 -prioritizeS3 -prioritizeEnum -angelixMaxSat
The groundtruth -- fixes by the developers, are in the groundtruth folder.
In each of the above folder, please find the folders t1 to t20, each of which contains the result for each of the 20 bugs that each tool generated. For example, if a repair tool generated a fix for a bug t1, then the fix could be found in t1/output/. This machine-generated fix can then be compared to the corresponding groundtruth at the file: groundtruth/t1/BLNote.txt.
====Results on the small programs================================================================
These programs along with the fixes generated by each repair tool can be found in 4 zip files: smallest_part1.zip to smallest_part4.zip
The fix for S3 for each program can be found in the folder named: angelixMeta213, in each of the program's folder. Again, look for the output folder, located at angelixMeta213/output, you can find the fix the generated by S3 for that program, if any. All bugs that are correctly fixed by S3 are reported in the file:
The fixes generated Enumerative, and CVC4 are at the folders named angelixEnum213, and angelixCVC4213 in each of the program's folder, respectively. All bugs that are correctly fixed by S3 are reported in the file
The fixes generated by Angelix, which are configured with multi-line change patch feature are in the folder named:
-group by suspiciousness score: angelixAngelix...sc213, where the ... is the group size which ranges from 1 to 4
All bugs fixed by angelix can be found in the file: (note that the file contains bugs fixed by many configuration of Angelix, but the largest number of bugs fixed by Angelix is 7 bugs, when the group size is set to 2)