RedundantListenerException when updating to mockito 3.2.4 with parallel test execution

[julius-d]

Hi!

I am trying to update a codebase to mockito 3.2.4!

After the update some tests fail with

org.mockito.exceptions.misusing.RedundantListenerException: 

Problems adding Mockito listener.
Listener of type 'MismatchReportingTestListener' has already been added and not removed.
It indicates that previous listener was not removed according to the API.
When you add a listener, don't forget to remove the listener afterwards:
  Mockito.framework().removeListener(myListener);
For more information, see the javadoc for RedundantListenerException class.

I was able to find out that the tests do not fail if I change the configuration of the maven-surefire-plugin to not run test in parallel.

The codebase uses
maven-surefire-plugin 2.22.1
maven-compiler-plugin 3.8.0
junit 4.12
java 1.8

I created an example project which reproduces the problem
https://github.com/julius-d/mockito-upgrade-problem
clone it and run mvn test
It will fail with RedundantListenerException
The tests do nothing special, they only waste time:

  @Test
  public void name() {
    IntStream.range(0,  Integer.MAX_VALUE).boxed()
        .forEach(i -> assertEquals(i, i));
  }

• The mockito message in the stacktrace have useful information, but it didn't help
• The problematic code (if that's possible) is copied here;
  Note that some configuration are impossible to mock via Mockito
• Provide versions (mockito / jdk / os / any other relevant information)
• Provide a Short, Self Contained, Correct (Compilable), Example of the issue
  (same as any question on stackoverflow.com)
• Read the contributing guide

[mockitoguy]

Thank you for reporting!

[mockitoguy]

What version you are migrating from?

[mockitoguy]

I dug into this issue here's what happens. When Maven Surefire plugin executes the tests with parallel "suitesAndClasses" feature I see an interesting symptom:

• one thread runs "withBefores" of the test (this is where the listener is created): https://github.com/mockito/mockito/blob/release/3.x/src/main/java/org/mockito/internal/runners/DefaultInternalRunner.java#L42
• different thread runs the actual test (this is where the listener is removed): https://github.com/mockito/mockito/blob/release/3.x/src/main/java/org/mockito/internal/runners/DefaultInternalRunner.java#L64

We expect the same thread to "start" the test and "finish" it. Internally, Mockito uses "ThreadLocal" to protect global state (global state is required to accommodate nice & sweet Mocking API).

What's next?

• try to fix MockitoJUnitRunner. This is not something easy to reproduce let alone fix. More over, JUnit Runner is old - now we use JUnit5 (JUnit Jupiter). Mockito team won't prioritize work on JUnit4 Runners. Take a stab at it if you feel strong! (I opened a PR that makes your repro repo a bit easier to work with: Updated the for easier debugging julius-d/mockito-upgrade-problem#1).
• workarounds:
  • use JUnit Rule instead of the Runner: public @Rule MockitoRule mockito = MockitoJUnit.rule().strictness(Strictness.STRICT_STUBS);. Rules are simpler, the JUnit Rules API is much cleaner.
  • use JUnit5.

Thoughts?

[julius-d]

Hi!

Thank you for taking a look :)
We use Mockito 2.25.1 at the moment, there we do not have the problem.

[gitrust]

I had the same issue with surefire (3.0.0-M5) parallel execution.
However it worked in my case when I configured surefire to run with <parallel>methods</parallel>.

[mdeinum]

Had the same issue when upgrading to 3.5.0 from 2.25.1. Changing the surefire <parallel> to methods fixed it. Leaving the default still makes it fail.

[sullis]

I encountered the same issue when migrating from Mockito 1.x to Mockito 3.5.11

My project uses JUnit 4.13 and the maven-surefire-plugin.

We utilize this Surefire configuration option:

      <parallel>classes</parallel>

I was able to eliminate the RedundantListenerException by changing classes to methods:

    <parallel>methods</parallel>

[Gr33nbl00d]

Any news to this?

[Gr33nbl00d]

I found out something very interesting:
I had some classes using MockitoRunner others were not using it (No Runner Defined).
The problem is gone when i let the other classes also use the MockitoRunner