mockStatic of enum class is unstable - impacted by earlier use of the class (unrelated to mockStatic)

[arivaldh]

OS: Windows 10
Mockito: 3.7.0
JUnit5: 5.7.0
Java: 8 (8u202)

We have two tests. A and B. When run in the order [B, A] they pass. When run in the order [A, B] B fails.
Failure is related to "switch" java statement with enum usage.

package com;

import org.junit.jupiter.api.MethodOrderer;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestMethodOrder;
import org.mockito.MockedStatic;
import org.springframework.test.util.ReflectionTestUtils;

import static com.MyEnum.A_VALUE;
import static com.MyEnum.B_VALUE;
import static com.MyEnum.C_VALUE;
import static org.assertj.core.api.Assertions.assertThat;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.mockStatic;
import static org.mockito.Mockito.when;

@TestMethodOrder(MethodOrderer.MethodName.class)
class MyFactoryClassWithEnumTest {
    @Test
    void aTest() {
        MyFactoryClass factory = MyFactoryClass.create();

        assertThat(factory.valuesVisible()).isEqualTo(new MyEnum[] {
                A_VALUE,
                B_VALUE,
                C_VALUE
        });
        factory.switchEval(A_VALUE);
    }

    @Test
    void bTest() {
        MyEnum mockInstance = mock(MyEnum.class);
        when(mockInstance.ordinal()).thenReturn(MyEnum.values().length);
        ReflectionTestUtils.setField(mockInstance, "ordinal", MyEnum.values().length);

        try (MockedStatic<MyEnum> mockedDropType = mockStatic(MyEnum.class)) {
            mockedDropType.when(MyEnum::values).thenReturn(new MyEnum[]{
                    A_VALUE,
                    B_VALUE,
                    C_VALUE,
                    mockInstance
            });

            MyFactoryClass factory = MyFactoryClass.create();
            assertThat(factory.valuesVisible()).isEqualTo(new MyEnum[] {
                    A_VALUE,
                    B_VALUE,
                    C_VALUE,
                    mockInstance});
            factory.switchEval(mockInstance);
        }
    }
}

While classes are as simple as possible:

package com;

public enum MyEnum {
    A_VALUE,
    B_VALUE,
    C_VALUE
}

package com;

import lombok.AllArgsConstructor;

@AllArgsConstructor(staticName = "create")
public class MyFactoryClass {
    public void switchEval(MyEnum myEnum) {
        switch (myEnum) {
            default:
                System.out.println("DONE!");
        }
    }

    public MyEnum[] valuesVisible() {
        return MyEnum.values();
    }
}

MyFactoryClass is the class that should return in (here absent) method "of" appropriate object related to enum value.
We want to test "default" scenario (if someone would be so nice to "override" our enum by his own enum by injecting it "earlier" in the classpath, or just when there's an incompatibility somewhere along the line).

Obviously if there's a better way to test such a scenario, we could change our approach accordingly.

[bdjelaili]

I think Mockito don't refresh the enum after calling mockStatic if it was aready loaded

[arivaldh]

I think Mockito don't refresh the enum after calling mockStatic if it was aready loaded

Could you clarify that you mean by that?
Do you know of a workaround for this problem?

I checked if Mockito.clearAllCaches() works. It wasn't working.
I'd also gladly use a different approach, if someone has one.

[bdjelaili]

Actualy I prefexed all my testes that use mockStatic like : xxxxWithStaticMockTest.java
and configure maven-surefire-plugin to do not reuseFork for these classes pattern

		<plugin>
			<groupId>org.apache.maven.plugins</groupId>
			<artifactId>maven-surefire-plugin</artifactId>
			<configuration>
				<skip>true</skip>
			</configuration>
			<!--	create a separate execution for classes that use org.mockito.Mockito.mockStatic
			mockStatic tests fails when using the same fork(VM) -->
			<executions>
				<execution>
					<id>allExceptMockStatic</id>
					<goals>
						<goal>test</goal>
					</goals>
					<configuration>
						<skip>false</skip>
						<excludes>
							<exclude>**/*WithStaticMockTest.java</exclude>
						</excludes>
					</configuration>
				</execution>
				<execution>
					<id>onlyMockStatic</id>
					<goals>
						<goal>test</goal>
					</goals>
					<configuration>
						<skip>false</skip>
						<includes>
							<exclude>**/*WithStaticMockTest.java</exclude>
						</includes>
						<forkCount>1</forkCount>
						<reuseForks>false</reuseForks>
					</configuration>
				</execution>
			</executions>
		</plugin>

[temp-droid]

Hey @arivaldh , could you share the exception/stack trace?

I'm wondering if ReflectionTestUtils.setField(mockInstance, "ordinal", MyEnum.values().length); couldn't be the reason your test fail. Because that call is not mockito related.

[temp-droid]

I tried to replicate your issue using:

• org.mockito:mockito-inline:3.7.0
• org.assertj:assertj-core:3.22.0
• org.junit.jupiter:junit-jupiter-api:5.7.0
• org.junit.jupiter:junit-jupiter-engine:5.7.0

but didn't succeed to reproduce the described behavior. Are you using mockito-core? Because that one fails on mocking the enum since it is final.

I got your run test running by simplifying it:

package com;

import org.junit.jupiter.api.MethodOrderer;
import org.junit.jupiter.api.Order;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestMethodOrder;
import org.mockito.MockedStatic;

import static com.MyEnum.A_VALUE;
import static com.MyEnum.B_VALUE;
import static com.MyEnum.C_VALUE;
import static org.assertj.core.api.Assertions.assertThat;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.mockStatic;

@TestMethodOrder(MethodOrderer.OrderAnnotation.class)
class MyFactoryClassWithEnumTest {
  @Order(0)
  @Test
  void aTest() {
    MyFactoryClass factory = MyFactoryClass.create();

    assertThat(factory.valuesVisible()).isEqualTo(new MyEnum[] {
        A_VALUE,
        B_VALUE,
        C_VALUE
    });
    factory.switchEval(A_VALUE);
  }

  @Order(1)
  @Test
  void bTest() {
    MyEnum mockInstance = mock(MyEnum.class);

    try (MockedStatic<MyEnum> mockedDropType = mockStatic(MyEnum.class)) {
      mockedDropType.when(MyEnum::values).thenReturn(new MyEnum[] {
          A_VALUE,
          B_VALUE,
          C_VALUE,
          mockInstance
      });

      MyFactoryClass factory = MyFactoryClass.create();
      assertThat(factory.valuesVisible()).isEqualTo(new MyEnum[] {
          A_VALUE,
          B_VALUE,
          C_VALUE,
          mockInstance
      });

      MyEnum myEnum = factory.switchEval(mockInstance);
      assertThat(myEnum).isEqualTo(mockInstance);
    }
  }
}

(I modified your switchEval method to return by default the enum that was passed.)

Another way to write this test would be to use your suggestion: 

if someone would be so nice to "override" our enum by his own enum by injecting it "earlier" in the classpath

We can do exactly that by creating a MyEnum in your test folder, for example:

package com;

public enum MyEnum {
    A_VALUE,
    B_VALUE,
    C_VALUE,
    D_VALUE
}

and the test:

package com;

import org.junit.jupiter.api.Test;

import static com.MyEnum.A_VALUE;
import static com.MyEnum.B_VALUE;
import static com.MyEnum.C_VALUE;
import static com.MyEnum.D_VALUE;
import static org.assertj.core.api.Assertions.assertThat;

class MyFactoryClassWithoutMockTest {
  @Test
  void cTest() {
    MyFactoryClass factory = MyFactoryClass.create();

    assertThat(factory.valuesVisible()).isEqualTo(new MyEnum[] {
        A_VALUE,
        B_VALUE,
        C_VALUE,
        D_VALUE });

    MyEnum myEnum = factory.switchEval(D_VALUE);
    assertThat(myEnum).isEqualTo(D_VALUE);
  }
}

[bdjelaili]

org.mockito:mockito-inline:4.0.0

public enum MyEnum {
    A_VALUE,
    B_VALUE,
    C_VALUE
}

public class MyFactoryClass {
	public void switchEval(MyEnum myEnum) {
		switch (myEnum) {
			default:
				System.out.println("DONE!");
		}
	}

	public MyEnum[] valuesVisible() {
		return MyEnum.values();
	}
}

import org.junit.jupiter.api.MethodOrderer;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestMethodOrder;
import org.mockito.MockedStatic;
import org.springframework.test.util.ReflectionTestUtils;

import static com.nxp.iot.device.management.service.MyEnum.A_VALUE;
import static com.nxp.iot.device.management.service.MyEnum.B_VALUE;
import static com.nxp.iot.device.management.service.MyEnum.C_VALUE;
import static org.assertj.core.api.Assertions.assertThat;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.mockStatic;
import static org.mockito.Mockito.when;

@TestMethodOrder(MethodOrderer.MethodName.class)
class MyFactoryClassWithEnumTest {
	@Test
	void aTest() {
		MyFactoryClass factory = new MyFactoryClass();

		assertThat(factory.valuesVisible()).isEqualTo(new MyEnum[] {
			A_VALUE,
			B_VALUE,
			C_VALUE
		});
		factory.switchEval(A_VALUE);
	}

	@Test
	void bTest() {
		int length = MyEnum.values().length;
		try (MockedStatic<MyEnum> mockedDropType = mockStatic(MyEnum.class)) {
			MyEnum mockInstance = mock(MyEnum.class);
			when(mockInstance.ordinal()).thenReturn(length);
			ReflectionTestUtils.setField(mockInstance, "ordinal", length);

			mockedDropType.when(MyEnum::values).thenReturn(new MyEnum[] {
				A_VALUE,
				B_VALUE,
				C_VALUE,
				mockInstance
			});

			MyFactoryClass factory = new MyFactoryClass();
			assertThat(factory.valuesVisible()).isEqualTo(new MyEnum[] {
				A_VALUE,
				B_VALUE,
				C_VALUE,
				mockInstance});
			factory.switchEval(mockInstance);
		}
	}
}

Execution :

java.lang.ArrayIndexOutOfBoundsException: 3

	at MyFactoryClass.switchEval(MyFactoryClass.java:3)
	at MyFactoryClassWithEnumTest.bTest(MyFactoryClassWithEnumTest.java:47)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:725)
	at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
	at org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)

[GregoryBevan]

I encountered the samed problem while mocking an Enum and found a workaround by moving the test that contains the MockedStatic call in an inner static class. This cause isolation of the test and doesn't interfere with other tests.

Not a solution, just a workaround...

[roli2otlet]

Still struggling with the same issue. Inner static class was not helping at all to us unfortunately.

[idovak]

Same issue reproduced on Linux and Win10, surfireplugin 2.22.2 java15. The solution/workaround bdjelaili commented on Jan13 helped me to overcome.

[idv-peratera]

Java: OpenJDK-18 (18.0.1), language level 11
Mockito: mockito-core, mockito-inline 4.10.0 (also built with 4.8.1)

Test:

    @Test
    @Order(value = 10)
    @SuppressWarnings(value = "ResultOfMethodCallIgnored")
    @DisplayName(value = "Case 10 :: Testing a login process with started process and unknown step (coverage only)")
    void givenInitLogin_whenLoginAlreadyStartedAndUnknownStep_thenExpectedAppropriateException() {
        //arrange
        final var nonexistentLoginStep = "NONEXISTENT";
        final var loginStepEnums = LoginStepsEnum.values();
        final var loginStepEnumsAdd = new LoginStepsEnum[loginStepEnums.length + 1];
        final var nonexistentLoginStepEnum = mock(LoginStepsEnum.class);
        when(nonexistentLoginStepEnum.ordinal()).thenReturn(loginStepEnums.length);
        System.arraycopy(loginStepEnums, 0, loginStepEnumsAdd, 0, loginStepEnums.length);
        ...
        //act&assert
        try (MockedStatic<LoginStepsEnum> loginStepEnum = Mockito.mockStatic(LoginStepsEnum.class)) {
            loginStepEnumsAdd[loginStepEnums.length] = nonexistentLoginStepEnum;
            loginStepEnum.when(LoginStepsEnum::values).thenReturn(loginStepEnumsAdd);
            loginStepEnum.when(() -> LoginStepsEnum.of(nonexistentLoginStep)).thenReturn(nonexistentLoginStepEnum);
            assertThrows(IllegalArgumentException.class, () -> authorizationService.initLogin(credentialsModel));
        }
    }

Enum:

enum LoginStepsEnum {

    STEP_1, STEP_2, STEP_3, STEP_4
}

Service code:

            ...
            final var processStep = LoginStepsEnum.of(processStepStr);
            if (processStep == null) {
                throw new CustomerException(...);
            }
            switch (processStep) {
                case STEP_1:
                    ...
                case STEP_2:
                    ...
                case STEP_3:
                    ...
                case STEP_4:
                    ...
                default:
                    throw new IllegalArgumentException("Invalid value for Login step");
            }
            ...

If I run just this single test, it's success, no any problems.
But on testing a test class with package of tests (while 10) I receive the exception:

java.lang.ArrayIndexOutOfBoundsException: Index 4 out of bounds for length 4

	at com.peratera.api.user.services.impl.AuthorizationServiceImpl.initLogin(AuthorizationServiceImpl.java:143)
	at com.peratera.api.user.services.impl.AuthorizationServiceImplTest.givenInitLogin_whenLoginAlreadyStartedAndUnknownStep_thenExpectedAppropriateException(AuthorizationServiceImplTest.java:375)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
	at java.base/java.lang.reflect.Method.invoke(Method.java:577)
	at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:686)
	at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
	at org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
	at org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:149)
	at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:140)
        ...

[pluttrell]

Does anyone know how to implement the temporary workaround provided by @bdjelaili using Gradle?

[LeN1Sensibl]

Hello
I'm still facing this issue.
Java 11
JUnit5
mockito 5.12.0
mockito-inline 5.2.0

I'm adding an enum value to test the default behaviour a a switch statement.
In standalone the test works, I can put an order to run it first in my test class but in a full module or maven test run, it always fails with the ArrayIndexOutOfBoundsException.
None of the workaround are working (inner or nested test classes, alone test in another class...)

would be nice to have someone looking at it.

Here is some simplified code for my enum

public enum MyEnum {
    VALUE_1,
    VALUE_2,
    ;

    public String toStringValue() {
        switch (this) {
            case VALUE_1:
                return "one";
            case VALUE_2:
                return "two";
            default:
                throw new IllegalArgumentException("not covered");
        }
    }
}

and the test

    void myEmnum_throwsException() {
        var originalValues = MyEnum.values();
        var size = originalValues.length;

        try (MockedStatic<MyEnum> mockedEnum = mockStatic(MyEnum.class)) {
            MyEnum UNSUPPORTED = mock(MyEnum.class);
            when(UNSUPPORTED.ordinal()).thenReturn(size);
            when(UNSUPPORTED.toStringValue()).thenCallRealMethod();

            var valuesWithUnsupported = Arrays.copyOf(originalValues, size + 1);
            valuesWithUnsupported[size] = UNSUPPORTED;
            mockedEnum.when(MyEnum::values).thenReturn(valuesWithUnsupported);

            assertThatIllegalArgumentException()
                    .isThrownBy(UNSUPPORTED::toStringValue)
                    .withMessage("not covered");
        }
    }

[vb-chris]

From an answer to a question on StackOverflow:

It is crucial that you run the code in the setup() Method before any class is loaded by the JVM classloader that contains a switch statement for the mocked Enum.

This may be another option to workaround the problem, provided that it's possible to control the order of tests.