Jmockit + infinitest not working since v1.26, v1.30 - StackOverflowError

[craftey]

Using IntelliJ's infinitest plugin I get the following exception:

java.lang.IllegalStateException: Duplicate application of the same mock-up class
        at mockit.MockUp.findPreviouslyMockedClassIfMockUpAlreadyApplied(MockUp.java:165)
        at mockit.MockUp.<init>(MockUp.java:119)
        at mockit.integration.testng.TestNGRunnerDecorator$MockParameter
        at mockit.integration.testng.TestNGRunnerDecorator.onExecutionStart(TestNGRunnerDecorator.java:349)
        at org.testng.TestNG.runExecutionListeners(TestNG.java:1110)
        at org.testng.TestNG.run(TestNG.java:1048)
        ...

Version 1.25 has no issues running the tests.
Version 1.26 produces the exception in line MockUp.java:165. Version 1.28 produces the exception in line MockUp.java:174., Version 1.29 produces the exception in MockUp.java:178.

Version 1.30 does not produce the exception. Tested with 2 different test-suites.
One test-suite successfully runs the tests with infinitest now.
Unfortunately the other test-suite produces many StackOverflowError in different places:

mockit.internal.state.TestRun:147 - StackOverflowError() - 14 tests in my suite fail with this exception
org.testng.internal.Parameters:-1 - StackOverflowError() - 4 tests fail like that
java.lang.ThreadLocal:419 - StackOverflowError() - 1 test
java.util.concurrent.ConcurentHashMap:936 - StackOverflowError() - 1 test

Running with mvn test or with the standard IntelliJ test runner works.
Configuring higher stack sizes for infinitest does not help, it just takes longer until the tests fail.
Playing arround with different Stacksizes -Xss1m or -Xss100m (configured in file infinitest.args) I figured, that in consecutive runs I get sometimes other Errors, meaning in the list above some errors were replaced by other errors:

mockit.internal.utilClassLoad:37 - StackOverflowError() - 1 test
sun.reflect.UnsafeFieldAccessorImpl:57 - StackOverflowError() - 1 test
mockit.integration.testng.TestNGRunnerDecorator:48 - StackOverflowError() - 1 test

A similar issue is documented here: hcoles/pitest#294

Best regards,
Craftey

[craftey]

I found a really weird workaround:

As I said jMockit version 1.26 to 1.29 let Infinitest fail every test with:

mockit.MockUp.java: IllegalStateException: Duplicate application of the same mock-up class

With jMockit version 1.30 only some tests fail, when run via Infinitest, with a StackoverflowError. (mvn test works always).

If I add to my configfile infinitest.filters the line

## groups=dummy

... the StackoverflowErrors go away.

This modification does not help with jMockit version 1.26 to 1.29.

For completness, my tests now run with Infinitest when I use jMockit version 1.30 and have the file

infintest.filters:

## TestNG Configuration
# Tests with a group-annotation in groups are the only ones being executed (overridden by excluded-groups). That is
# what the docu says. Unfortunately Infinitest seems to ignore the groups setup. I have tested to mark a test with
# that group "dummy", but Infinitest still executes all tests. But, another positive side effect of the
# following line is that it fixes an Infinitest bug that leads to StackoverflowErrors in conjunction with jmockit v 1.30.
## groups=dummy
# Tests with a group-annotation in excluded-groups are not executed:
## excluded-groups=remote,resourceDependent,selenium,slow
.*IT
.*SSC

placed in my project root folder.

The line ## groups=dummy seems to be the deciding bit o_O.

[StefanPenndorf]

@rliesenfeld
I had a quick look at TestNGRunnerDecorator it has a subclass MockParameters which is a MockUp<Parameters> and has a static method getInjectedParameter with the same signature than Parameters from TestNG.

I didn't dive into the JMockit code but as far as I understand the comments from MockUp class there is some byte code / JVM magic in place that redirects all calls to an original (even static) method (here: TestNG's Parameters.getInjectedParameter()) to the MockUp class (here: TestNGRunnerDecorator$MockParameters.getInjectedParameter()).

But your implementation of TestNGRunnerDecorator$MockParameters.getInjectedParameter() calls Parameters.getInjectedParameter() which seems to be redirected to TestNGRunnerDecorator$MockParameters.getInjectedParameter() which calls ....

I'm wondering whether a MockUp implementation may call it's original implementation. Are there any guards in place?

[craftey]

I have created a minimal example to reproduce the issue with jmockit+dataprovider+inifinitest:
We need 3 files: 2 test classes one with dataprovider and a pom with jmockit, testng, mockito.
A part of junit seems to be needed for infinitest. Adding mockito-core adds the missing class.

TestWithDataProvider.java:

package example;
import org.testng.annotations.DataProvider;
import org.testng.annotations.Test;

public class TestWithDataProvider {

    @DataProvider
    private Object[][] data() {
        return new Object[][] { { "foo" } };
    }

    @Test(dataProvider = "data")
    public void minimalDataProviderTest(String param) {
    }
}

OtherTest.java:

package example;
import org.testng.annotations.Test;

public class OtherTest {

    @Test
    public void minimalTest() {
    }
}

pom.xml:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>de.mri</groupId>
    <artifactId>jmockit-dataprovider-example</artifactId>
    <version>1.0-SNAPSHOT</version>

    <properties>
        <maven.compiler.target>1.8</maven.compiler.target>
        <maven.compiler.source>1.8</maven.compiler.source>
        <encoding>UTF-8</encoding>
        <project.build.sourceEncoding>${encoding}</project.build.sourceEncoding>
    </properties>

    <dependencies>
        <dependency>
            <groupId>org.testng</groupId>
            <artifactId>testng</artifactId>
            <version>6.9.10</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.mockito</groupId>
            <artifactId>mockito-core</artifactId>
            <version>1.10.19</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.jmockit</groupId>
            <artifactId>jmockit</artifactId>
            <version>1.30</version>
            <scope>test</scope>
        </dependency>
    </dependencies>
</project>

• Running the tests with mvn clean test works.
• Running the tests with IntelliJs plugin Infinitest fails with Stackoverflow error, when jmockit version 1.30.
• Running the tests with Infinitest fails with IllegalStateException: Duplicate application of the same mock-up class, when jmockit version 1.26-1.29.
• Running the tests with Infinitest and jmockit 1.25 works.
• Running the tests with Infinitest and jmockit 1.26+ and commented @DataProvider works, but then we do not use DataProvider.

@rliesenfeld Can you do anything about the issue on jmockits side or do we have to fix infinitest or testng?

[rliesenfeld]

I closed this issue because it a) did not provide information for me to reproduce the problem, b) didn't look as something caused by a bug in JMockit, but rather some unexpected interaction with other tools (maybe TestNG, maybe Infinitest, maybe something else), and c) it looks like it would be a really painful exercise to try and figure this out, for the benefit of a tool combination that I don't use.

I think this is the kind of issue that somebody else should try and fix (if a PR is submitted, I will give it due consideration). The user community shouldn't expect the project owner to solve every problem. (Just like, for example, Mockito developers don't work on Android- or Scala-specific issues.)

[craftey]

@rliesenfeld Thanks you for your reply.

In reply to a): What about the minimal example from #379 (comment) to reproduce the issue?

Reply to b) I know its an interaction between 3 tools. But, as I observed, concrete versions of jmockit, while keeping the other tools unchanged, lead to different outcomes with the minimal test. Outcomes from "works" to "IllegalStateException" to "Stackoverflow error"

Reply to c) Maybe the IllegalStateException when using jmockit 1.26-1.29 is easier to track down.
Other tools like pitest also had issues starting with jmockit 1.26 (hcoles/pitest#294). So maybe this issue is more than a rare combination of tools.

@KyleRogers Is the issue still present with pitest+jmockit?

I would like to have this issue fixed because currently infinitest + jmockit + dataprovider does not work but it used to work very nice 2-3 month ago. So finally I would like to ask: Who else should I ask to look at this issue? How should I act wisely to get towards a solution?

[craftey]

Starting from jmockit version 1.36 this issue is fixed somehow. But also a newer version of TestNG (6.14.3) fixes this issue with older versions of jmockit.

https://github.com/craftey/infinitest-issues/tree/master/testng-jmockit-issue