Fault localization process fails when using CocoSpoonBasedSpectrumBasedFaultLocalizer

[DehengYang]

Sorry for disturbing you again with a new issue...

Previously, I used GZoltarFaultLocalizer to generate a rank list of suspicious statements when using Nopol to repair real bugs from defects4j.

This time, I tried CocoSpoonBasedSpectrumBasedFaultLocalizer, which is also in the same folder of GZoltarFaultLocalizer (i.e., src/main/java/fr/inria/lille/localization), to localize faults. However, the error happens during running the tests. The detailed error info is:

================================= The Nopol is repairing Chart_3 now =================================

Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF8
java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.util.concurrent.TimeoutException
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:202)
at fr.inria.lille.repair.Main.main(Main.java:98)
Caused by: java.lang.RuntimeException: java.util.concurrent.TimeoutException
at xxl.java.junit.TestSuiteExecution.executionResult(TestSuiteExecution.java:106)
at xxl.java.junit.TestSuiteExecution.runTest(TestSuiteExecution.java:39)
at fr.inria.lille.localization.CocoSpoonBasedSpectrumBasedFaultLocalizer.runTests(CocoSpoonBasedSpectrumBasedFaultLocalizer.java:50)
at fr.inria.lille.localization.DumbFaultLocalizerImpl.(DumbFaultLocalizerImpl.java:30)
at fr.inria.lille.localization.CocoSpoonBasedSpectrumBasedFaultLocalizer.(CocoSpoonBasedSpectrumBasedFaultLocalizer.java:33)
at fr.inria.lille.repair.nopol.NoPol.createLocalizer(NoPol.java:198)
at fr.inria.lille.repair.nopol.NoPol.build(NoPol.java:138)
at fr.inria.lille.repair.Main$1.call(Main.java:93)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.util.concurrent.TimeoutException
at java.util.concurrent.FutureTask.get(FutureTask.java:201)
at xxl.java.junit.TestSuiteExecution.executionResult(TestSuiteExecution.java:99)
... 11 more

And as is shown in the log file:

06:47:54.955 [pool-2-thread-1] DEBUG xxl.java.junit.TestSuiteExecution - Timeout after 5 seconds. Infinite loop?

I have made two attempts to study the reason behind the error:
1) I use Nopol with CocoSpoonBasedSpectrumBasedFaultLocalizer to repair the given example in the folder /Nopol/test-projects, and the both the fault localization and repair process are successful (the expected patch is got). Therefore, I thought this error only occurs when using CocoSpoonBasedSpectrumBasedFaultLocalizer to repair real bugs in defects4j.

To verify this thought, I use Nopol with CocoSpoonBasedSpectrumBasedFaultLocalizer to repair some bugs (e.g., chart 3, math 2, math 58), all of which get errors as mentioned above.

2) By reviewing previous log files when using GZoltarFaultLocalizer , I found the same log info DEBUG xxl.java.junit.TestSuiteExecution - Timeout after 5 seconds. Infinite loop? commonly exists in previous logs that records how Nopol with GZoltarFaultLocalizer repair real bugs (e.g., math 58) in defects4j. But these scenarios using GZoltarFaultLocalizer all successfully generate suspicious list of statements.

Instead, the scenarios using CocoSpoonBasedSpectrumBasedFaultLocalizer stop and report TimeoutException.

---

Much appreciated if any help is given.

[monperrus]

Thanks for the report. Can you provide us with a failing test case? (see test cases in nopl-core/src/test)

[DehengYang]

I am really sorry for my delay. I was occupied with a friend's wedding these days so that I did not make any further attempts... Sorry again for my delay in my replying.

The current version of nopol I am using is the version proposed in March 2017 (I have been using this version for a long time, thus I don't upgrade the nopol version to the newest):

Based on this version, I created a test case in file CocospoonLocalizerTest.java:

package fr.inria.lille.localization;

import fr.inria.lille.localization.metric.Ochiai;
import fr.inria.lille.repair.common.config.NopolContext;
import fr.inria.lille.repair.nopol.SourceLocation;
import org.junit.Assert;
import org.junit.Test;

import java.io.File;
import java.net.URL;
import java.util.List;
import java.util.Map;

import java.util.*; //added by dale.

import static gov.nasa.jpf.util.test.TestJPF.assertEquals;
import static gov.nasa.jpf.util.test.TestJPF.assertTrue;

/**
 * Created by bdanglot on 10/4/16.
 */
public class CocospoonLocalizerTest {

    @Test
    public void testOchiaiCoCoSpoonLocalizer() throws Exception {

		/* test OchiaiCoCoSpoonLocalizer : the SourceLocation must be sorted following the Ochiai metric */

        File[] sources = new File[]{new File("/home/apr/environment/buggyPrograms/NopolRepairPrograms/Math/Math_58/src/main/java")};
	String tests="org.apache.commons.math.distribution.HypergeometricDistributionTest";
        URL[] classpath = new URL[]{
                new File("/home/apr/environment/buggyPrograms/NopolRepairPrograms/Math/Math_58/target/classes/").toURI().toURL(),
                new File("/home/apr/environment/buggyPrograms/NopolRepairPrograms/Math/Math_58/target/test-classes/").toURI().toURL()
        };
        //String[] testClasses = new String[]{"org.apache.commons.math.optimization.fitting.GaussianFitterTest"};
	String[] testClasses = tests.split(", ");
        CocoSpoonBasedSpectrumBasedFaultLocalizer localizer = new CocoSpoonBasedSpectrumBasedFaultLocalizer(new NopolContext(sources, classpath, testClasses), new Ochiai());
        Map<SourceLocation, List<TestResult>> executedSourceLocationPerTest = localizer.getTestListPerStatement();
        ///assertEquals(8, executedSourceLocationPerTest.keySet().size());

        SourceLocation sourceLocation1 = new SourceLocation("org.apache.commons.math.optimization.general.LevenbergMarquardtOptimizer", 741);


	System.out.println("dale stmt 0:"+executedSourceLocationPerTest.size()+"   "+new ArrayList<>(executedSourceLocationPerTest.keySet()).get(0) +new ArrayList<>(executedSourceLocationPerTest.keySet()).get(0).getSuspiciousValue()  );
        assertTrue(executedSourceLocationPerTest.keySet().contains(sourceLocation1));


        List<SourceLocation> sortedStatements = new ArrayList<>(executedSourceLocationPerTest.keySet());

        assertEquals(1, sortedStatements.get(0).getSuspiciousValue(), 10E-3);


        //assertEquals(sourceLocation5, sortedStatements.get(0).getLocation());
    }

}

The I rebuilt nopol with mvn clean package -DskipTests and ran this test using mvn test -Dtest=CocospoonLocalizerTest. The test info is:

---

T E S T S

Running fr.inria.lille.localization.CocospoonLocalizerTest
current testMethod to be tested is org.apache.commons.math.distribution.HypergeometricDistributionTest#testDegenerateNoFailures
current testMethod to be tested is org.apache.commons.math.distribution.HypergeometricDistributionTest#testDegenerateNoSuccesses
current testMethod to be tested is org.apache.commons.math.distribution.HypergeometricDistributionTest#testDegenerateFullSample
current testMethod to be tested is org.apache.commons.math.distribution.HypergeometricDistributionTest#testPreconditions
current testMethod to be tested is org.apache.commons.math.distribution.HypergeometricDistributionTest#testLargeValues
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 16.592 sec <<< FAILURE!
testOchiaiCoCoSpoonLocalizer(fr.inria.lille.localization.CocospoonLocalizerTest) Time elapsed: 16.53 sec <<< ERROR!
java.lang.RuntimeException: java.util.concurrent.TimeoutException
at java.util.concurrent.FutureTask.get(FutureTask.java:201)
at xxl.java.junit.TestSuiteExecution.executionResult(TestSuiteExecution.java:99)
at xxl.java.junit.TestSuiteExecution.runTest(TestSuiteExecution.java:39)
at fr.inria.lille.localization.CocoSpoonBasedSpectrumBasedFaultLocalizer.runTests(CocoSpoonBasedSpectrumBasedFaultLocalizer.java:51)
at fr.inria.lille.localization.DumbFaultLocalizerImpl.(DumbFaultLocalizerImpl.java:30)
at fr.inria.lille.localization.CocoSpoonBasedSpectrumBasedFaultLocalizer.(CocoSpoonBasedSpectrumBasedFaultLocalizer.java:33)
at fr.inria.lille.localization.CocospoonLocalizerTest.testOchiaiCoCoSpoonLocalizer(CocospoonLocalizerTest.java:37)

Results :

Tests in error:
CocospoonLocalizerTest.testOchiaiCoCoSpoonLocalizer:37 » Runtime java.util.con...

Tests run: 1, Failures: 0, Errors: 1, Skipped: 0

[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 21.160 s
[INFO] Finished at: 2019-01-30T18:25:21-08:00
[INFO] Final Memory: 22M/300M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.14.1:test (default-test) on project nopol: There are test failures.

I think the reason is that cocospoon localizer failed on running the test: org.apache.commons.math.distribution.HypergeometricDistributionTest#testLargeValues. But I still have no idea how to fix this.

After that, I checked if the newest cocospoon java file:https://github.com/SpoonLabs/nopol/blob/master/nopol/src/main/java/fr/inria/lille/localization/CocoSpoonBasedSpectrumBasedFaultLocalizer.java can fix this problem. But it doesn't work when I transform some code from this newest file to the old file, after which the nopol can be built successfully but still cannot pass the cocospoon test.

Any help would be really appreciated.

Thank you again for your great patience and kindness!

[monperrus]

Could you create a pull request with only the failing test case? Thanks.

[DehengYang]

I have forked the nopol master branch into my github repo and then update the CocospoonLocalizerTest.java (#180). I then create a pull request with only the failing test case, with comments as follows:

This test aims to expose the java.lang.RuntimeException when running test cases in Math_58, a real bug from defects4j. But this test file (CocospoonLocalizerTest.java) is created based on the version of Nopol of March 2017.

More details can be seen at #180. (But in this website, the information All checks have failed is shown. This does not suprise me as the added test case is built based on the old nopol version(March 2017), But I cannot find that version in the github repo of Nopol.)

Thank you for your continuous help!

[DehengYang]

I am sorry, do I misunderstand your guidance (create a pull request with only the failing test case)? Should I create a new java file (e.g., called CocospoonLocalizer2Test.java) rather than directly modifying the current CocospoonLocalizerTest.java. I am not sure about that. Thanks!

[monperrus]

Closed, see #182