Skip to content
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
docs
results
script generate contingency tables Feb 20, 2019
.gitignore
README.md Update README.md May 29, 2019

README.md

RepairThemAll Experiment

This repository contains the raw results of the execution of 11 repair tools on 5 bug benchmarks presented in the following paper:

@inproceedings{RepairThemAll2019,
  author    = {Thomas Durieux and Fernanda Madeiral and Matias Martinez and Rui Abreu},
  title     = {{Empirical Review of Java Program Repair Tools: A Large-Scale Experiment on 2,141 Bugs and 23,551 Repair Attempts}},
  booktitle = {Proceedings of the 27th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE '19)},
  year      = {2019},
  url       = {https://arxiv.org/abs/1905.11973}
}

The execution framework that has been used is available at: https://github.com/program-repair/RepairThemAll

The data of this repository is also available as a website: http://program-repair.org/RepairThemAll_experiment

Repository Structure

The repository is structured as follow:

- docs: content for the website
- results: 
-- <benchmark>
--- <project>
---- <bug_id>
----- <repair tool>
------<seed>
------- grid5k.stderr.log  stderr of the execution
------- grid5k.stdout.log stdout of the execution (without the repair)
------- repair.log repair log
------- result.json sandardize output 
------- detailed_result.json raw output of the repair tool if it generates a json file
- script
-- get_patched_bugs.py the script that are used to generate the table for the paper

Patches Bugs

# Benchmark Bug Repair Tool
1 Bears Rules 55669767 2 Arja Kali
2 Bears dubbo 29773929 2 Cardumen NPEFix
3 Bears openhtmltopdf 46994292 3 Cardumen jGenProg Kali
4 Bears libra 36524727 2 Arja Kali
5 Bears vavr 61240331 2 Arja Kali
6 Bears p6spy 31527545 1 Arja
7 Bears jsonignore 42015798 1 jGenProg
8 Bears commons 93313389 1 Kali
9 Bears commons 63513297 1 Kali
10 Bears AutomatedCar 51759763 4 Arja GenProg Kali RSRepair
11 Bears traccar 88474474 5 Arja Cardumen jGenProg jKali Kali
12 Bears traccar 95455832 6 Arja Cardumen jGenProg jKali jMutRepair Kali
13 Bears traccar 98488474 4 Arja Cardumen jGenProg jKali
14 Bears traccar 12061222 6 Arja Cardumen jGenProg jKali jMutRepair Kali
15 Bears traccar 16167822 1 jGenProg
16 Bears traccar 37846623 1 Cardumen
17 Bears traccar 55052458 5 Arja Cardumen jGenProg jKali Kali
18 Bears traccar 68530427 4 Cardumen jGenProg jKali jMutRepair
19 Bears traccar 69490884 6 Arja Cardumen jGenProg jKali jMutRepair Kali
20 Bears traccar 73136493 6 Arja Cardumen jGenProg jKali jMutRepair Kali
21 Bears traccar 05713931 1 Kali
22 Bears traccar 08872339 1 Kali
23 Bears traccar 16639485 4 Cardumen jGenProg jKali jMutRepair
24 Bears traccar 22428590 4 Cardumen jGenProg jKali jMutRepair
25 Bears la4j 36911083 1 Nopol
26 Bugs.jar Accumulo 2fd7633f 1 DynaMoth
27 Bugs.jar Accumulo 3d55560a 4 jGenProg jKali jMutRepair Kali
28 Bugs.jar Accumulo 6dbbdc21 1 Kali
29 Bugs.jar Accumulo 023be574 1 DynaMoth
30 Bugs.jar Accumulo 72fd6bec 1 Cardumen
31 Bugs.jar Accumulo 73ce9cfb 2 DynaMoth Nopol
32 Bugs.jar Accumulo 94c2a31f 2 jGenProg Kali
33 Bugs.jar Accumulo 872b6db3 1 DynaMoth
34 Bugs.jar Accumulo 994df698 2 DynaMoth Nopol
35 Bugs.jar Accumulo 7651b777 1 DynaMoth
36 Bugs.jar Accumulo 9339ecf8 1 Nopol
37 Bugs.jar Accumulo 15476a0d 2 Arja Kali
38 Bugs.jar Accumulo 813109d7 1 Arja
39 Bugs.jar Accumulo a450ac2f 2 DynaMoth Nopol
40 Bugs.jar Accumulo a64151e6 2 Arja Kali
41 Bugs.jar Accumulo b007b22e 2 Arja Kali
42 Bugs.jar Accumulo cd7feb4d 1 Kali
43 Bugs.jar Accumulo d9ab8449 1 DynaMoth
44 Bugs.jar Accumulo df4b1985 2 DynaMoth Nopol
45 Bugs.jar Accumulo f99b5654 2 DynaMoth Nopol
46 Bugs.jar Accumulo f848178e 1 DynaMoth
47 Bugs.jar Math 0d057fc6 1 DynaMoth
48 Bugs.jar Math 2a6c6409 5 Arja GenProg jMutRepair Kali Nopol
49 Bugs.jar Math 2a9cbbab 1 DynaMoth
50 Bugs.jar Math 2b852d79 1 DynaMoth
51 Bugs.jar Math 2c8a114f 1 DynaMoth
52 Bugs.jar Math 3a08bfa6 2 DynaMoth Nopol
53 Bugs.jar Math 3a15d8ce 1 DynaMoth
54 Bugs.jar Math 3c4cb189 2 DynaMoth Nopol
55 Bugs.jar Math 3f645310 2 DynaMoth Nopol
56 Bugs.jar Math 4ebd967c 1 DynaMoth
57 Bugs.jar Math 5b9302d5 2 DynaMoth Nopol
58 Bugs.jar Math 5e638976 1 DynaMoth
59 Bugs.jar Math 5fe9b36c 2 Cardumen jGenProg
60 Bugs.jar Math 6eb46555 1 DynaMoth
61 Bugs.jar Math 8a83581e 2 DynaMoth Nopol
62 Bugs.jar Math 8dd22390 1 RSRepair
63 Bugs.jar Math 8e5867ed 2 Cardumen GenProg
64 Bugs.jar Math 9aabf587 1 DynaMoth
65 Bugs.jar Math 9e0c5ad4 7 Cardumen GenProg jGenProg jKali jMutRepair Kali RSRepair
66 Bugs.jar Math 19c1c3bb 1 DynaMoth
67 Bugs.jar Math 26a61077 3 DynaMoth jMutRepair Nopol
68 Bugs.jar Math 32b0f733 1 DynaMoth
69 Bugs.jar Math 48dde378 1 DynaMoth
70 Bugs.jar Math 59a0da9c 3 Arja DynaMoth Nopol
71 Bugs.jar Math 061f5017 1 DynaMoth
72 Bugs.jar Math 63a48705 2 DynaMoth Nopol
73 Bugs.jar Math 66dece12 2 DynaMoth Nopol
74 Bugs.jar Math 83f18d52 2 DynaMoth Nopol
75 Bugs.jar Math 91d280b7 1 DynaMoth
76 Bugs.jar Math 95d15eff 2 DynaMoth Nopol
77 Bugs.jar Math 97b440fc 1 DynaMoth
78 Bugs.jar Math 118e94b5 2 DynaMoth Nopol
79 Bugs.jar Math 118f0cc0 2 DynaMoth Nopol
80 Bugs.jar Math 185e3033 1 DynaMoth
81 Bugs.jar Math 334c01e6 1 DynaMoth
82 Bugs.jar Math 350f726c 2 DynaMoth Nopol
83 Bugs.jar Math 424cbd20 1 DynaMoth
84 Bugs.jar Math 495f04bc 1 Cardumen
85 Bugs.jar Math 596ccd59 7 Arja Cardumen GenProg jKali jMutRepair Kali RSRepair
86 Bugs.jar Math 0596e314 2 DynaMoth Nopol
87 Bugs.jar Math 607c9ec6 2 DynaMoth Nopol
88 Bugs.jar Math 621bbb8f 2 DynaMoth Nopol
89 Bugs.jar Math 645d642b 2 DynaMoth Nopol
90 Bugs.jar Math 784e4f69 1 jGenProg
91 Bugs.jar Math 1352a70f 2 DynaMoth Nopol
92 Bugs.jar Math 2123f780 2 Arja jGenProg
93 Bugs.jar Math 2836a6f9 1 DynaMoth
94 Bugs.jar Math 6844aba9 2 DynaMoth Nopol
95 Bugs.jar Math 7980a242 2 DynaMoth Nopol
96 Bugs.jar Math 7994d3ee 2 DynaMoth Nopol
97 Bugs.jar Math 38983e82 2 DynaMoth Nopol
98 Bugs.jar Math 49444ee6 1 DynaMoth
99 Bugs.jar Math 69273dca 4 Arja GenProg jGenProg RSRepair
100 Bugs.jar Math 86545dab 1 DynaMoth
101 Bugs.jar Math 328513f3 1 Arja
102 Bugs.jar Math 724795b5 1 DynaMoth
103 Bugs.jar Math 73605560 1 DynaMoth
104 Bugs.jar Math a4ffd393 1 DynaMoth
105 Bugs.jar Math a06a1584 7 Arja GenProg jGenProg jKali jMutRepair Kali RSRepair
106 Bugs.jar Math a6f96306 1 Nopol
107 Bugs.jar Math a49e443c 1 DynaMoth
108 Bugs.jar Math a56d4998 1 Cardumen
109 Bugs.jar Math a7363a2a 6 DynaMoth GenProg jKali jMutRepair Kali Nopol
110 Bugs.jar Math abe53a53 2 DynaMoth Nopol
111 Bugs.jar Math ac597cc1 1 DynaMoth
112 Bugs.jar Math ad252a8c 1 DynaMoth
113 Bugs.jar Math b01fcc31 2 DynaMoth Nopol
114 Bugs.jar Math b2e24119 2 DynaMoth Nopol
115 Bugs.jar Math b6bf8f41 1 jMutRepair
116 Bugs.jar Math b07ecae3 1 DynaMoth
117 Bugs.jar Math b9ca51f0 2 DynaMoth Nopol
118 Bugs.jar Math b55e0206 2 DynaMoth Nopol
119 Bugs.jar Math bda25b40 1 DynaMoth
120 Bugs.jar Math bfbb156d 2 DynaMoth Nopol
121 Bugs.jar Math c0b49542 1 DynaMoth
122 Bugs.jar Math c44bfe00 1 jGenProg
123 Bugs.jar Math c979a6f0 1 DynaMoth
124 Bugs.jar Math cc4ab51e 8 Arja Cardumen DynaMoth jGenProg jKali jMutRepair Kali Nopol
125 Bugs.jar Math ce126bdb 2 DynaMoth Nopol
126 Bugs.jar Math cedf0d27 1 DynaMoth
127 Bugs.jar Math d4f978dd 1 Nopol
128 Bugs.jar Math d7c0f27e 2 DynaMoth Nopol
129 Bugs.jar Math d270055e 1 DynaMoth
130 Bugs.jar Math d2777388 2 DynaMoth Nopol
131 Bugs.jar Math dbdff075 2 DynaMoth Nopol
132 Bugs.jar Math dd6cefb0 2 DynaMoth Nopol
133 Bugs.jar Math e2dc384d 1 Arja
134 Bugs.jar Math ebadb558 2 DynaMoth Nopol
135 Bugs.jar Math ebc61de9 1 DynaMoth
136 Bugs.jar Math ef9b639a 2 DynaMoth Nopol
137 Bugs.jar Math f4a4464b 1 NPEFix
138 Bugs.jar Math f6dd42b4 6 Arja Cardumen GenProg jGenProg Nopol RSRepair
139 Bugs.jar Math f64b6a90 2 DynaMoth Nopol
140 Bugs.jar Math f656676e 2 DynaMoth Nopol
141 Bugs.jar Math faa77857 2 DynaMoth Nopol
142 Bugs.jar Math faf99727 8 Arja DynaMoth GenProg jGenProg jKali jMutRepair Kali Nopol
143 Bugs.jar Math fc409e88 1 Cardumen
144 Bugs.jar Flink 0cfa43d7 1 Arja
145 Bugs.jar Flink 5dfc897b 2 Kali Nopol
146 Bugs.jar Flink 6ecd0f82 1 DynaMoth
147 Bugs.jar Flink 8dc70f2e 2 jGenProg jMutRepair
148 Bugs.jar Flink 32a003d5 2 DynaMoth Nopol
149 Bugs.jar Flink 45fb6d82 2 Arja DynaMoth
150 Bugs.jar Flink 259f10c0 2 DynaMoth Nopol
151 Bugs.jar Flink 380ef878 1 DynaMoth
152 Bugs.jar Flink 5308ac83 1 Nopol
153 Bugs.jar Flink adb321d6 2 DynaMoth Nopol
154 Bugs.jar Oak 00df38d2 1 DynaMoth
155 Bugs.jar Oak 0fa892b3 1 Kali
156 Bugs.jar Oak 1bf5c550 3 Cardumen DynaMoth Nopol
157 Bugs.jar Oak 01f5a26f 1 NPEFix
158 Bugs.jar Oak 3ce758b7 3 Arja jKali Kali
159 Bugs.jar Oak 3f51fb09 1 Nopol
160 Bugs.jar Oak 7c320b1e 3 DynaMoth Kali Nopol
161 Bugs.jar Oak 36e70bd7 1 DynaMoth
162 Bugs.jar Oak 93c1aa40 2 DynaMoth Nopol
163 Bugs.jar Oak 591e4d4a 1 DynaMoth
164 Bugs.jar Oak 668f08f2 1 NPEFix
165 Bugs.jar Oak 3270e761 1 DynaMoth
166 Bugs.jar Oak 5449bf39 2 Cardumen jGenProg
167 Bugs.jar Oak 717186d6 2 DynaMoth Nopol
168 Bugs.jar Oak 5286861d 2 Arja Kali
169 Bugs.jar Oak ba38c380 1 DynaMoth
170 Bugs.jar Oak c83755c3 1 Kali
171 Bugs.jar Oak e115fd90 1 Nopol
172 Bugs.jar Oak f2a2edec 1 Nopol
173 Bugs.jar Oak ffa818f3 4 Arja DynaMoth Kali Nopol
174 Bugs.jar Log4J2 0bea17d7 1 DynaMoth
175 Bugs.jar Log4J2 2afe3dff 1 Nopol
176 Bugs.jar Log4J2 3b4b370e 1 DynaMoth
177 Bugs.jar Log4J2 3eb44094 1 Nopol
178 Bugs.jar Log4J2 4cf831b6 1 DynaMoth
179 Bugs.jar Log4J2 7bb1ad47 2 DynaMoth Nopol
180 Bugs.jar Log4J2 16ad8763 1 DynaMoth
181 Bugs.jar Log4J2 86d8944f 1 DynaMoth
182 Bugs.jar Log4J2 837dcd89 1 DynaMoth
183 Bugs.jar Log4J2 73400bfb 1 DynaMoth
184 Bugs.jar Log4J2 97203de8 1 DynaMoth
185 Bugs.jar Log4J2 a523dcd5 1 DynaMoth
186 Bugs.jar Log4J2 afcf92eb 1 Nopol
187 Bugs.jar Log4J2 b2ec5106 2 DynaMoth Nopol
188 Bugs.jar Log4J2 c8fd3c53 1 DynaMoth
189 Bugs.jar Log4J2 d8af1c93 1 Nopol
190 Bugs.jar Maven c4002945 2 Arja Kali
191 Bugs.jar Wicket 0f8a6d75 1 DynaMoth
192 Bugs.jar Wicket 2fc6a395 3 DynaMoth Kali Nopol
193 Bugs.jar Wicket a4a3a9a6 1 DynaMoth
194 Bugs.jar Wicket a382917f 1 DynaMoth
195 Bugs.jar Wicket b80f6640 1 DynaMoth
196 Bugs.jar Wicket b672cb2d 1 DynaMoth
197 Bugs.jar Wicket bcea89fc 1 DynaMoth
198 Bugs.jar Wicket e93fdd5a 1 DynaMoth
199 Bugs.jar Wicket f3ec1503 1 DynaMoth
200 Bugs.jar Wicket f539c18c 1 Kali
201 Defects4J Chart 1 9 Arja Cardumen DynaMoth GenProg jGenProg jKali jMutRepair Kali RSRepair
202 Defects4J Chart 3 4 Arja GenProg jGenProg RSRepair
203 Defects4J Chart 4 2 Nopol NPEFix
204 Defects4J Chart 5 7 DynaMoth GenProg jGenProg jKali Kali Nopol RSRepair
205 Defects4J Chart 6 1 Cardumen
206 Defects4J Chart 7 4 Arja jGenProg jMutRepair RSRepair
207 Defects4J Chart 8 1 DynaMoth
208 Defects4J Chart 9 1 Nopol
209 Defects4J Chart 11 1 Cardumen
210 Defects4J Chart 12 4 Arja GenProg Kali RSRepair
211 Defects4J Chart 13 8 Arja Cardumen DynaMoth GenProg jGenProg jKali Nopol RSRepair
212 Defects4J Chart 14 2 Arja NPEFix
213 Defects4J Chart 15 6 Arja DynaMoth jGenProg jKali Kali RSRepair
214 Defects4J Chart 16 1 NPEFix
215 Defects4J Chart 17 1 Nopol
216 Defects4J Chart 18 1 DynaMoth
217 Defects4J Chart 19 1 Arja
218 Defects4J Chart 21 1 Nopol
219 Defects4J Chart 24 1 Cardumen
220 Defects4J Chart 25 10 Arja DynaMoth GenProg jGenProg jKali jMutRepair Kali Nopol NPEFix RSRepair
221 Defects4J Chart 26 2 Kali NPEFix
222 Defects4J Closure 1 5 Arja DynaMoth GenProg Kali Nopol
223 Defects4J Closure 2 2 DynaMoth Nopol
224 Defects4J Closure 3 4 Arja DynaMoth Kali Nopol
225 Defects4J Closure 4 2 DynaMoth Nopol
226 Defects4J Closure 5 2 Kali Nopol
227 Defects4J Closure 6 1 Nopol
228 Defects4J Closure 7 4 Arja DynaMoth Kali Nopol
229 Defects4J Closure 8 4 Arja Kali Nopol RSRepair
230 Defects4J Closure 9 1 Nopol
231 Defects4J Closure 10 7 Arja GenProg jKali jMutRepair Kali Nopol RSRepair
232 Defects4J Closure 11 2 DynaMoth Nopol
233 Defects4J Closure 12 4 Arja DynaMoth Kali Nopol
234 Defects4J Closure 13 3 DynaMoth jKali Nopol
235 Defects4J Closure 14 1 Nopol
236 Defects4J Closure 15 3 Arja Kali Nopol
237 Defects4J Closure 16 1 Nopol
238 Defects4J Closure 17 2 DynaMoth Nopol
239 Defects4J Closure 18 1 Nopol
240 Defects4J Closure 19 2 DynaMoth Nopol
241 Defects4J Closure 20 4 Arja DynaMoth Kali Nopol
242 Defects4J Closure 21 9 Arja DynaMoth GenProg jGenProg jKali jMutRepair Kali Nopol RSRepair
243 Defects4J Closure 22 9 Arja DynaMoth GenProg jGenProg jKali jMutRepair Kali Nopol RSRepair
244 Defects4J Closure 23 2 DynaMoth Nopol
245 Defects4J Closure 24 2 DynaMoth Nopol
246 Defects4J Closure 25 2 DynaMoth Nopol
247 Defects4J Closure 26 3 DynaMoth Kali Nopol
248 Defects4J Closure 27 2 DynaMoth Nopol
249 Defects4J Closure 28 2 DynaMoth Nopol
250 Defects4J Closure 29 3 DynaMoth Kali Nopol
251 Defects4J Closure 30 2 Kali Nopol
252 Defects4J Closure 31 5 Arja DynaMoth Kali Nopol RSRepair
253 Defects4J Closure 32 2 DynaMoth Nopol
254 Defects4J Closure 33 6 Arja DynaMoth GenProg Kali Nopol RSRepair
255 Defects4J Closure 34 1 Nopol
256 Defects4J Closure 35 2 DynaMoth Nopol
257 Defects4J Closure 36 3 DynaMoth Kali Nopol
258 Defects4J Closure 37 1 Nopol
259 Defects4J Closure 38 1 Nopol
260 Defects4J Closure 39 1 Nopol
261 Defects4J Closure 40 1 Nopol
262 Defects4J Closure 41 1 Nopol
263 Defects4J Closure 42 1 Nopol
264 Defects4J Closure 43 1 Nopol
265 Defects4J Closure 44 1 Nopol
266 Defects4J Closure 45 6 Arja DynaMoth jKali Kali Nopol RSRepair
267 Defects4J Closure 46 6 Arja DynaMoth GenProg jKali Kali Nopol
268 Defects4J Closure 48 2 DynaMoth Nopol
269 Defects4J Closure 49 1 Kali
270 Defects4J Closure 50 2 Kali Nopol
271 Defects4J Closure 51 1 Nopol
272 Defects4J Closure 52 1 Nopol
273 Defects4J Closure 53 2 DynaMoth Nopol
274 Defects4J Closure 54 1 Nopol
275 Defects4J Closure 55 5 Arja GenProg Kali Nopol RSRepair
276 Defects4J Closure 56 1 Nopol
277 Defects4J Closure 57 1 Nopol
278 Defects4J Closure 58 1 Nopol
279 Defects4J Closure 59 7 Arja DynaMoth GenProg jKali Kali Nopol RSRepair
280 Defects4J Closure 60 2 DynaMoth Nopol
281 Defects4J Closure 61 3 Kali Nopol RSRepair
282 Defects4J Closure 62 5 DynaMoth jKali jMutRepair Kali Nopol
283 Defects4J Closure 63 5 DynaMoth jKali jMutRepair Kali Nopol
284 Defects4J Closure 64 2 DynaMoth Kali
285 Defects4J Closure 67 4 Arja GenProg Kali RSRepair
286 Defects4J Closure 68 3 Arja DynaMoth Kali
287 Defects4J Closure 70 1 DynaMoth
288 Defects4J Closure 71 1 DynaMoth
289 Defects4J Closure 72 2 DynaMoth Kali
290 Defects4J Closure 75 5 Arja DynaMoth GenProg Kali RSRepair
291 Defects4J Closure 76 2 DynaMoth Kali
292 Defects4J Closure 77 1 DynaMoth
293 Defects4J Closure 78 5 Arja DynaMoth GenProg Kali RSRepair
294 Defects4J Closure 109 1 DynaMoth
295 Defects4J Closure 111 1 DynaMoth
296 Defects4J Closure 112 4 Arja GenProg Kali RSRepair
297 Defects4J Closure 114 4 Arja GenProg Kali RSRepair
298 Defects4J Closure 115 8 Arja DynaMoth GenProg jGenProg jKali jMutRepair Kali RSRepair
299 Defects4J Closure 116 1 Arja
300 Defects4J Closure 117 4 Arja GenProg Kali RSRepair
301 Defects4J Closure 118 4 Arja GenProg Kali RSRepair
302 Defects4J Closure 119 4 Arja GenProg Kali RSRepair
303 Defects4J Closure 120 3 Arja Kali RSRepair
304 Defects4J Closure 121 3 Arja Kali RSRepair
305 Defects4J Closure 122 1 Kali
306 Defects4J Closure 124 4 Arja GenProg Kali RSRepair
307 Defects4J Closure 125 4 Arja DynaMoth Kali Nopol
308 Defects4J Closure 126 9 Arja DynaMoth GenProg jGenProg jKali jMutRepair Kali Nopol RSRepair
309 Defects4J Closure 127 5 Arja GenProg Kali Nopol RSRepair
310 Defects4J Closure 128 1 Nopol
311 Defects4J Closure 129 7 Arja DynaMoth GenProg jKali Kali Nopol RSRepair
312 Defects4J Closure 130 5 Arja GenProg Kali Nopol RSRepair
313 Defects4J Closure 131 3 Kali Nopol RSRepair
314 Defects4J Closure 132 3 Arja Kali Nopol
315 Defects4J Closure 133 3 Arja Kali Nopol
316 Defects4J Lang 7 3 Arja GenProg RSRepair
317 Defects4J Lang 16 1 Arja
318 Defects4J Lang 20 1 Arja
319 Defects4J Lang 22 1 Arja
320 Defects4J Lang 27 1 Arja
321 Defects4J Lang 39 1 Arja
322 Defects4J Lang 43 1 Arja
323 Defects4J Lang 44 2 Kali Nopol
324 Defects4J Lang 45 2 Arja RSRepair
325 Defects4J Lang 46 1 Nopol
326 Defects4J Lang 50 1 Arja
327 Defects4J Lang 51 5 Arja DynaMoth Kali Nopol RSRepair
328 Defects4J Lang 53 1 Nopol
329 Defects4J Lang 55 3 Arja GenProg Nopol
330 Defects4J Lang 58 2 Arja Nopol
331 Defects4J Lang 59 1 Arja
332 Defects4J Lang 60 1 Arja
333 Defects4J Lang 61 1 Arja
334 Defects4J Lang 63 3 Arja DynaMoth RSRepair
335 Defects4J Math 2 8 Arja Cardumen DynaMoth GenProg jGenProg Kali Nopol RSRepair
336 Defects4J Math 4 1 Nopol
337 Defects4J Math 5 3 Arja jGenProg RSRepair
338 Defects4J Math 7 1 Nopol
339 Defects4J Math 8 8 Arja Cardumen DynaMoth GenProg jGenProg jKali Kali RSRepair
340 Defects4J Math 18 1 Nopol
341 Defects4J Math 20 4 DynaMoth jGenProg jKali Nopol
342 Defects4J Math 22 2 Arja GenProg
343 Defects4J Math 24 1 Nopol
344 Defects4J Math 28 9 Arja Cardumen DynaMoth GenProg jGenProg jKali jMutRepair Kali RSRepair
345 Defects4J Math 31 4 Arja GenProg Kali RSRepair
346 Defects4J Math 32 3 DynaMoth jKali Kali
347 Defects4J Math 33 1 Nopol
348 Defects4J Math 39 2 Arja RSRepair
349 Defects4J Math 40 2 GenProg jGenProg
350 Defects4J Math 41 2 DynaMoth Nopol
351 Defects4J Math 42 1 Nopol
352 Defects4J Math 49 9 Arja Cardumen DynaMoth GenProg jGenProg jKali Kali Nopol RSRepair
353 Defects4J Math 50 9 Arja Cardumen DynaMoth GenProg jGenProg jMutRepair Kali Nopol RSRepair
354 Defects4J Math 53 3 Arja jGenProg RSRepair
355 Defects4J Math 56 3 Arja jGenProg RSRepair
356 Defects4J Math 57 1 jMutRepair
357 Defects4J Math 58 5 Arja DynaMoth jMutRepair Nopol RSRepair
358 Defects4J Math 60 3 Cardumen jGenProg RSRepair
359 Defects4J Math 63 1 Cardumen
360 Defects4J Math 64 1 jGenProg
361 Defects4J Math 68 1 RSRepair
362 Defects4J Math 69 1 Nopol
363 Defects4J Math 70 6 Arja Cardumen GenProg jGenProg NPEFix RSRepair
364 Defects4J Math 71 3 Arja DynaMoth RSRepair
365 Defects4J Math 73 6 Arja Cardumen GenProg jGenProg Nopol RSRepair
366 Defects4J Math 74 1 RSRepair
367 Defects4J Math 76 1 Nopol
368 Defects4J Math 78 1 Nopol
369 Defects4J Math 80 10 Arja Cardumen DynaMoth GenProg jGenProg jKali jMutRepair Kali Nopol RSRepair
370 Defects4J Math 81 9 Arja DynaMoth GenProg jGenProg jKali jMutRepair Kali Nopol RSRepair
371 Defects4J Math 82 6 Arja DynaMoth jGenProg jKali jMutRepair Nopol
372 Defects4J Math 84 6 Arja jGenProg jKali jMutRepair Kali RSRepair
373 Defects4J Math 85 10 Arja Cardumen DynaMoth GenProg jGenProg jKali jMutRepair Kali Nopol RSRepair
374 Defects4J Math 87 1 Nopol
375 Defects4J Math 88 2 jMutRepair Nopol
376 Defects4J Math 91 1 NPEFix
377 Defects4J Math 95 6 Arja Cardumen GenProg jGenProg Kali RSRepair
378 Defects4J Math 98 2 Arja NPEFix
379 Defects4J Math 103 3 Arja GenProg RSRepair
380 Defects4J Math 105 2 DynaMoth Nopol
381 Defects4J Mockito 6 1 NPEFix
382 Defects4J Mockito 8 1 DynaMoth
383 Defects4J Mockito 10 2 Arja Kali
384 Defects4J Time 11 1 DynaMoth
385 Defects4J Time 14 1 Nopol
386 IntroClassJava checksum 9111_005 1 Arja
387 IntroClassJava digits 8b7c_005 1 DynaMoth
388 IntroClassJava digits 363f_000 1 jMutRepair
389 IntroClassJava digits aaca_004 1 Nopol
390 IntroClassJava digits 36a6_000 1 jMutRepair
391 IntroClassJava digits 96d2_004 2 jGenProg RSRepair
392 IntroClassJava digits 540f_004 2 Arja RSRepair
393 IntroClassJava digits 393c_002 3 Arja GenProg RSRepair
394 IntroClassJava digits beb4_003 8 Arja DynaMoth GenProg jGenProg jKali Kali Nopol RSRepair
395 IntroClassJava digits 3516_001 2 Arja GenProg
396 IntroClassJava digits 070c_000 7 Arja DynaMoth GenProg jGenProg jKali Kali RSRepair
397 IntroClassJava digits 66dc_000 1 jMutRepair
398 IntroClassJava digits 66dc_002 1 jMutRepair
399 IntroClassJava digits 9d9a_002 2 Arja RSRepair
400 IntroClassJava grade bf82_001 2 jMutRepair Nopol
401 IntroClassJava grade bf82_003 2 jMutRepair Nopol
402 IntroClassJava grade 07ce_000 1 jMutRepair
403 IntroClassJava grade 07ce_001 1 jMutRepair
404 IntroClassJava median 8b7c_003 3 Arja GenProg RSRepair
405 IntroClassJava median 3ff6_003 1 Nopol
406 IntroClassJava median 8ee9_000 4 Arja GenProg Nopol RSRepair
407 IntroClassJava median 69ac_003 2 jMutRepair Nopol
408 IntroClassJava median 7829_003 2 jMutRepair Nopol
409 IntroClassJava median 7829_006 2 GenProg RSRepair
410 IntroClassJava median 708f_007 1 Nopol
411 IntroClassJava median 4c62_000 2 jMutRepair Nopol
412 IntroClassJava median aaca_003 3 Arja jMutRepair Nopol
413 IntroClassJava median a389_000 2 jMutRepair Nopol
414 IntroClassJava median 36a6_000 1 jGenProg
415 IntroClassJava median 474f_010 2 GenProg RSRepair
416 IntroClassJava median 96d2_000 1 Nopol
417 IntroClassJava median 678b_012 1 Nopol
418 IntroClassJava median 678b_015 1 Nopol
419 IntroClassJava median 540f_000 3 Arja GenProg RSRepair
420 IntroClassJava median 540f_002 3 Arja GenProg RSRepair
421 IntroClassJava median 540f_003 1 Nopol
422 IntroClassJava median 17f7_010 4 Arja GenProg Nopol RSRepair
423 IntroClassJava median 5144_010 2 jMutRepair Nopol
424 IntroClassJava median 9e93_000 3 Arja jMutRepair Nopol
425 IntroClassJava median 6e92_001 1 Nopol
426 IntroClassJava smallest 8ee9_003 2 Arja RSRepair
427 IntroClassJava smallest 7829_007 7 Arja DynaMoth GenProg jKali Kali Nopol RSRepair
428 IntroClassJava smallest 7829_008 2 jMutRepair Nopol
429 IntroClassJava smallest 4c62_001 2 GenProg RSRepair
430 IntroClassJava smallest 24fa_007 2 jMutRepair Nopol
431 IntroClassJava smallest a389_003 2 jMutRepair Nopol
432 IntroClassJava smallest 36a6_001 2 jMutRepair Nopol
433 IntroClassJava smallest 6ea5_000 2 jMutRepair Nopol
434 IntroClassJava smallest 4fe6_003 2 jMutRepair Nopol
435 IntroClassJava smallest bc7b_005 1 Arja
436 IntroClassJava smallest b98b_007 1 Arja
437 IntroClassJava smallest b98b_009 2 Arja GenProg
438 IntroClassJava smallest 8b71_007 1 RSRepair
439 IntroClassJava smallest 841a_002 7 Arja DynaMoth GenProg jKali Kali Nopol RSRepair
440 IntroClassJava smallest 841a_003 7 Arja DynaMoth GenProg jKali Kali Nopol RSRepair
441 IntroClassJava smallest 17f7_005 1 RSRepair
442 IntroClassJava smallest c4ae_000 1 GenProg
443 IntroClassJava smallest 2912_000 2 Arja RSRepair
444 IntroClassJava smallest 3b11_000 1 GenProg
445 IntroClassJava smallest b70d_000 2 jMutRepair Nopol
446 IntroClassJava smallest 98a7_003 4 Arja jMutRepair Nopol RSRepair
447 IntroClassJava smallest 6d0e_000 2 jMutRepair Nopol
448 QuixBugs DEPTH_FIRST_SEARCH 8 Arja Cardumen DynaMoth GenProg jGenProg jKali Kali RSRepair
449 QuixBugs DETECT_CYCLE 1 NPEFix
450 QuixBugs HANOI 1 NPEFix
451 QuixBugs KNAPSACK 1 jMutRepair
452 QuixBugs LEVENSHTEIN 1 Cardumen
453 QuixBugs LIS 4 Arja GenProg jGenProg RSRepair
454 QuixBugs MERGESORT 1 Cardumen
455 QuixBugs NEXT_PERMUTATION 1 jMutRepair
456 QuixBugs QUICKSORT 7 Arja DynaMoth jKali jMutRepair Kali Nopol RSRepair
457 QuixBugs RPN_EVAL 1 Cardumen
458 QuixBugs SHORTEST_PATH_LENGTHS 4 Arja GenProg jGenProg RSRepair
459 QuixBugs SQRT 1 GenProg
Repair Tools Bears Bugs.jar Defects4J IntroClassJava QuixBugs Total
Arja 12 (4%) 21 (1%) 86 (21%) 23 (7%) 4 (10%) 146 (6%)
GenProg-A 1 (0%) 9 (0%) 45 (11%) 18 (6%) 4 (10%) 77 (3%)
Kali-A 15 (5%) 24 (2%) 72 (18%) 5 (1%) 2 (5%) 118 (5%)
RSRepair-A 1 (0%) 6 (0%) 62 (15%) 22 (7%) 4 (10%) 95 (4%)
Cardumen 13 (5%) 12 (1%) 17 (4%) 0 (0%) 4 (10%) 46 (2%)
jGenProg 13 (5%) 14 (1%) 31 (7%) 4 (1%) 3 (7%) 65 (3%)
jKali 10 (3%) 8 (0%) 27 (6%) 5 (1%) 2 (5%) 52 (2%)
jMutRepair 7 (2%) 11 (0%) 20 (5%) 24 (8%) 3 (7%) 65 (3%)
Nopol 1 (0%) 72 (6%) 107 (27%) 32 (10%) 1 (2%) 213 (9%)
DynaMoth 0 (0%) 124 (10%) 74 (18%) 6 (2%) 2 (5%) 206 (9%)
NPEFix 1 (0%) 3 (0%) 9 (2%) 0 (0%) 2 (5%) 15 (0%)
Total 74 304 550 139 31 1098

Chi-square Test of independence

# Patched # Non-Patched
Arja on Defects 86 309
Arja on Others 60 1686
Chi2 value= 170.43487132271886 p-value= 5.945480330471514e-39 Degrees of freedom= 1
# Patched # Non-Patched
GenProg on Defects 45 350
GenProg on Others 32 1714
Chi2 value= 84.90652479289551 p-value= 3.128091736130167e-20 Degrees of freedom= 1
# Patched # Non-Patched
Kali on Defects 72 323
Kali on Others 46 1700
Chi2 value= 150.4020168750391 p-value= 1.4160845009256217e-34 Degrees of freedom= 1
# Patched # Non-Patched
RSRepair on Defects 62 333
RSRepair on Others 33 1713
Chi2 value= 144.80217516680622 p-value= 2.372523759882535e-33 Degrees of freedom= 1
# Patched # Non-Patched
Cardumen on Defects 17 378
Cardumen on Others 29 1717
Chi2 value= 10.701973234378928 p-value= 0.0010702132907778191 Degrees of freedom= 1
# Patched # Non-Patched
jGenProg on Defects 31 364
jGenProg on Others 34 1712
Chi2 value= 38.10114926497659 p-value= 6.717055566199569e-10 Degrees of freedom= 1
# Patched # Non-Patched
jKali on Defects 27 368
jKali on Others 25 1721
Chi2 value= 39.69012031778793 p-value= 2.976273080413384e-10 Degrees of freedom= 1
# Patched # Non-Patched
jMutRepair on Defects 20 375
jMutRepair on Others 45 1701
Chi2 value= 6.76253623850222 p-value= 0.009309135821381086 Degrees of freedom= 1
# Patched # Non-Patched
Nopol on Defects 107 288
Nopol on Others 106 1640
Chi2 value= 158.83167769741897 p-value= 2.036659201530019e-36 Degrees of freedom= 1
# Patched # Non-Patched
DynaMoth on Defects 74 321
DynaMoth on Others 132 1614
Chi2 value= 46.25197019724452 p-value= 1.0398193445599021e-11 Degrees of freedom= 1
# Patched # Non-Patched
NPEFix on Defects 9 386
NPEFix on Others 6 1740
Chi2 value= 17.333764012540335 p-value= 3.1356574417361234e-05 Degrees of freedom= 1

Total generated patch: 67211

Execution time 314 days, 12:29:19.419491

You can’t perform that action at this time.