Using Spies (and Fakes)

This page describes a few (common) scenarios where Mockito spies work effectively.

Invariants and State

Mock object can be a great tool if used properly. But when the test double has invariants to be respected, mocking isn't most effective at this kind of job.

Case Study 1: to test a servlet that uses HttpServletRequest

As demonstrated in Mocks are Stupid, HttpServletRequest is one such object with invariants:

HttpServletRequest reqStub = mock(HttpServletRequest.class);
when(reqStub.getParameterMap()).thenReturn(ImmutableMap.of(key, new String[]{val}));

Mocking or stubbing getParameterMap() causes brittle tests because there are several different ways in which the subject under test's implementation could interact with the request object. It could call request.getParameterMap().get(key), or request.getParameterValues(key).

Typically, test doubles with invariants are better created as fakes: proper Java classes that override methods with behavior.

The main problem with fakes is that they require more code than you may want to write: HttpServletRequest has like 30 or 40 methods to implement, among which only 2 or 3 methods are relevant to the test.

Mockito's @Spy allows the test to only implement the 2 or 3 methods of interest and leave the rest abstract:

@Spy private FakeHttpServletRequest request;

@Test public void testBadAttributeCausesAutoLogout() {
  request.setAttribute(MAGIC_ATTRIBUTE_KEY, "bad");
  new LoginServlet().service(request, response);
  verify(request).logout();
}

static abstract class FakeHttpServletRequest implements HttpServletRequest {
  private final Map<String, Object> attributes = new LinkedHashMap<>();

  @Override public Object getAttribute(String name) {
    return attributes.get(name);
  }

  @Override public Enumeration<String> getAttributeNames() {
    return Iterators.asEnumeration(attributes.keySet().iterator());
  }

  @Override public void setAttribute(String name, Object value) {
    attributes.put(name, value);
  }
}

Not only stubbing state, the above test can still use verify(request).logout() to verify interactions: logout() be called once and only once.

Case Study 2: to test a JobScheduler that uses ScheduledExecutorService

Another example where @Spy helps to create better test code: suppose we have a Job scheduler framework that internally utilizes a ScheduledExecutorService to invoke the jobs at the right time.

To test such framework, manually programing the ScheduledExecutorService.schedule() and submit() methods and the time-related logic would be tedious, at best.

The following test uses @Spy to create a relatively trivial FakeScheduledExecutorService and then uses it to test behaviors, for instance: a job failed with SERVER_TOO_BUSY gets rescheduled for a later run:

public class JobSchdulerTest {
  @Spy private FakeClock clock;
  @Spy private FakeScheduledExecutorService executor;
  @Mock private Task task;

  @Test public void testJobsFailedWithTooBusyGetsRescheduled() {
    scheduler().add(task, Duration.ofMillis(10));
    elapse(Duration.ofMillis(9));
    // should not have run
    verify(task, never()).run();

    // at scheduled time, it runs, but failed with TOO_BUSY
    when(task.run()).thenReturn(SERVER_TOO_BUSY);
    elapse(Duration.ofMillis(1));
    verify(task).run();
    reset(task);

    // so it's retried.
    when(task.run()).thenReturn(OK);
    elapse(Duration.ofMillis(20));
    verify(task).run();
    reset(task);

    // Retry succeeded. No more runs.
    elapse(Duration.ofMillis(10000));
    verify(task, never()).run();
  }

  private JobScheduler scheduler() {
    return new JobScheduler(clock, executor);
  }

  // Moves time and invokes ready jobs.
  private void elapse(Duration time) {
    clock.elapse(time);
    executor.run();
  }

  static abstract class FakeClock extends Clock {
    private Instant now = Instant.ofEpochMilli(0);

    @Override public Instant instant() {
      return now;
    }

    void elapse(Duration duration) {
      now = now.plus(duration);
    }
  }

  abstract class FakeScheduledExecutorService
      implements ScheduledExecutorService {
    private List<Job> jobs = new ArrayList<>();

    @Override public ScheduledFuture<?> schedule(
        Runnable command, long delay, TimeUnit unit) {
      jobs.add(new Job(command, delay, unit));
    }

    /** Runs all jobs that are ready by now. Leaves the rest. */
    void run() {
      Instant now = clock.instant();
      List<Job> ready = jobs.stream().filter(job -> job.ready(now)).collect(toList());
      jobs = jobs.stream()
          .filter(job -> job.pending(now))
          .collect(toCollection(ArrayList::new));
      ready.forEach(Job::run);
    }
  }
}

While it requires a bit of code in the FakeScheduledExecutorService class, we get to extract orthogonal logic out of the tests so that tests stay clean and easy to understand. In reality, the fake tends to be reused by many different tests in the test class, so it more than pays for itself.

It's worth noting that the class being spied is allowed to be non-static innner class of the test class, which enables it to read state from other fields (in this case, the clock object). Using this technique, we make the executor and the clock working together seamlessly.

• Do implement object invariants with spies.
• Do not mock() or when().thenReturn() invariants.

Dummy objects

A somewhat common mistake is as reported in this Stack Overflow thread. That is, trying to use a factory helper that returns a mock while configuring another mock. The code can look quite innocent and puzzling:

Model model = mock(Model.class);
when(model.getSubModel()).thenReturn(dummySubModel());

private SubModel dummySubModel() {
  SubModel sub = mock(SubModel.class);
  when(sub.getName()).thenReturn("anything but null");
  // other dummy states...
  return sub;
}

What happens essentially if dummySubModel() were inlined looks like this:

Model model = mock(Model.class);
when(model.getSubModel());
SubModel sub = mock(SubModel.class);
when(sub.getName());  // Oops!

the second when() call happened before the first when() call is finished.

For helpers that create dummy objects like this, using spy() minimizes caveats and surprises down the road when other people inevitably attempt to use your helper method, because there is no when() call involved (just @Override's):

Model model = mock(Model.class);
when(model.getSubModel()).thenReturn(dummySubModel());

private SubModel dummySubModel() {
  return spy(DummySubModel.class);
}

static abstract class DummySubModel implements SubModel {
  @Override public String getName() {
    return "anything but null";
  }
  // other dummy states...
}

Or, since dummies tend to be stateless anyway, might as well just skip the dummySubModel() helper method and declare it as a @Spy field:

@Spy private DummySubModel subModel;

...
Model model = mock(Model.class);
when(model.getSubModel()).thenReturn(subModel);
...

static abstract class DummySubModel implements SubModel {
  @Override public String getName() {
    return "anything but null";
  }
}

• Do use spy() or @Spy to create dummies
• Do not return a mock from helper methods.
• Do not use when().thenReturn() to program dummies.

Default behavior

It isn't uncommon to stub a method with default behavior in setUp():

@Mock private UserService userService;

@Before public void setUp() {
  doAnswer(new Answer<Object>() {
    @Override public Object answer(InvocationOnMock invocationOnMock) {
      @SuppressWarnings("unchecked")  // It's static type declared by the method signature.
      List<User> users = (List<User>) invocationOnMock.getArguments()[0];
      Policy policy = (Policy) invocationOnMock.getArguments()[1];
      assertThat(users).isNotEmpty();
      ...
      return true;
    }
  }).when(userService).addUsers(Matchers.<List<User>>any(), any(Policy.class));
}

While it works, the code isn't too readable or maintainable:

• The casts! Unchecked!
• The method signature and the implementation read backwards.
• The need of the matchers. Look at that Matchers.<List<User>>any().

Instead, consider to use an abstract class with @Spy:

@Spy private StubUserService userService;

// Don't need the setUp() any more.

static abstract class StubUserService implements UserService {
  @Override public boolean addUsers(List<User> users, Policy policy) {
    assertThat(users).isNotEmpty();
    ...
    return true;
  }
}

And again, verify(userService).addUsers(...) is supported; when(userService.addUsers(...) is supported when needed.

• Do use spy() or @Spy on an abstract class to implement the default behavior.
• Do declare method parameters with the correct types and use them away.
• Do not doAnswer() in a setUp() or @Before method.
• Do not cast.

Matchers for List elements.

Mockito supports parameter matchers so for example one can write when(userService.addUser(notNull())).

But when the parameter is a List<User> instead, you can't do when(userService.addUser(asList(noNull(), eq(user)))).

@Spy can be used as a workaround:

@Spy private MockUserService userService;

@Test public void testSecondUser() {
  User user = ...;
  when(userService.doAddUsers(notNull(), eq(user))).thenReturn(false);
  ...
}

static abstract class MockUserService implements UserService {
  @Override public boolean addUsers(List<User> users, Policy policy) {
    return doAddUsers(users.toArray(new User[0]));
  }

  abstract boolean doAddUsers(User... users);
}

The abstract class MockUserService declares a matcher friendly varargs method doAddUsers(User...) and delegates to it. The tests can then use the matcher friendly abstract method to stub and verify.

As a general rule of thumb, the MockUserService code should be self-evidently correct. Be careful not to add non-trivial logic to the delegating code because conditionals and complex logic in tests can lead to false-negative tests (tests that pass for the wrong reason).