Mockito initialise performance

[keiki85]

Hello it's me again :-),

after upgrading our big project from 1 to mockito 3 we have no performance problem, if all unit tests are run.

Yet in the daily developing work we recognised that current mockito needs round about 3 times the time to initialise Mockito itself.
On my laptop with an i7-10850H (6 normal, 6 HT, 5.1 GHz single max) CPU I see following stats for a simple Mockito.mock in a junit test.

Mockito 1: ~ 100ms
Mockito 3.2.14: 320-360 ms

A bit of profiling showed me that default plugin loading and class generation for mockito itself seems to be causing most of the time. Yet for me it's difficult to know or see the big picture here.

I created a simple project to see the difference for yourself.
mockito-performance.zip

Maybe we can get closer to the earlier timings?

Thanks in advance

[npepinpe]

Just to chime in, we also recently updated to 3.12.4 and ran into this. It can take up to 2s on a beefy machine to initialize mockito. Below is a flamegraph with a profiler - note that the profile slows things down a bit, so timing may vary, but without it running any single test takes at least 1.5s, up to 2s, when the actual test takes 3-5ms. It's fine in CI since this is only once, but it's a bit annoying during active development 🙂

To be fair, I realize the issue is there since earlier (downgrading to any 3.x version didn't help), and in general it's worth it for the value the library brings - but it would be nice if initialization was fast, so that when we run a single test while developing it's also fast 😄

[TimvdLippe]

We have had previous reports about performance problems and we shipped a reset API after #2178 Would that help in your case?

[keiki85]

Thanks @TimvdLippe for the fast reply. I know the other ticket, where I am the thread creator. Yet in my understanding the other ticket and reset API tackles a general performance problem overall. Maybe we had a misunderstanding here?

Here I am talking about the performance of one simple Mockito.mock(simpleInterface). Which is quite expensive as Mockito needs to initialise itself first. After the initialisation was done I consider Mockito fast.
And the initialisation of itself is quite expensive. As I said on my fast laptop it's only over 300 ms a simple Mockito.mock. On worse hardware of my colleagues it's way worse.

[casid]

@TimvdLippe Problem here is TDD roundtrip time. For instance, take the test below that mocks a single interface with one method in it. This test takes about 1 second to run on my Macbook Pro 2015. On the same machine I can run an entire test suite with over thousand unit tests in a project that uses hand-written test doubles. This has quite an impact on productivity, especially with short TDD roundtrips.

import static org.mockito.Mockito.mock;

import java.io.Closeable;
import java.util.concurrent.TimeUnit;

import org.junit.jupiter.api.Test;


public class PerformanceTest {

   @Test
   void mockedInterface() {
      long start = System.nanoTime();
      mock(Closeable.class);
      long end = System.nanoTime();

      System.out.println("Took " + TimeUnit.NANOSECONDS.toMillis(end - start) + "ms");
   }

}

[jzy3d]

Thanks for your example @casid. Facing the same kind of issue, I resolved with manual mocking.