Skip to content

Comparison of TypeScript mocking frameworks

Jump to bottom
Hadrien Milano edited this page Mar 7, 2020 · 5 revisions

This document compares different mocking libraries for TypeScript found on npm.

Overview

Key: good points in bold, bad points in italic, neutral in normal font.

Library safe-mock OmniMock Substitute ts-mockery ts-mockito
Requirements
TypeScript version >= 2.1 >= 3.1 >=3.0 unknown unknown
Requires Proxy Yes Yes Yes No depends3
Types of mock
Backed mocks No Yes No Yes No
Virtual mocks Yes Yes Yes Yes Yes
 Function mocks Yes Yes No Yes No
Stubbing features
Method/function return value Yes Yes Yes Yes Yes
Method/function throw Yes Yes No No Yes
Stub members/getters No Yes No Yes Yes
Call forwarding No Yes No No No
Call fake No Yes Yes Yes partial4
Deep mocks No Yes No members only No
Verification
Unexpected calls partial Yes No Yes Yes
Missing calls Yes Yes No Yes Yes
Argument matchers No Yes Yes partial2 partial5
Verify setters No No Yes No No
Type safety
Undefined behavior unknown No unknown unknown Yes
Return values Yes Yes Somewhat1 Yes Yes
Function arguments No Yes Yes Yes Yes
Templated functions Yes No No Yes Yes
Developper experience
Mock is the stub Yes No Yes Yes No
Error messages - - - - -

1Because of the way it deals with the setter-as-stub design, Substitute requires you turn off strict null checking, which some may consider an essential feature of TypeScript.
2Matchers only available for verification, not for expectation setting. This can be worked around by writing some logic in a fake. 3Proxy is only required for virtual mocks. 4The prototype of the original method is completely lost. The fake method is effectively typed as (...any[]) => any. 5Matchers have type any

Features

This section explains the features presented in the main table.

Requires Proxy

The Proxy object is a key API for reflectivity in JavaScript. Virtual mocks cannot be achieved without the Proxy API, and the Proxy API can not be polyfilled. You will not be able to use a library relying on Proxy if your tests have to run in an environment which does not support it.

Backed mocks

Backed mocks are constructed from an object which exists at runtime. That object becomes the backing instance of the mock and allows features which cannot be achieved with virtual mocks, such as call forwarding and enumerability of the object's property.

Virtual mocks

Virtual mocks are created from a type only. This lets you mock a class without calling its constructor, or even just an interface.
Virtual mocks can exhibit strange behaviour at runtime because type information is lost at runtime. Besides, virtual mocks can only be implemented using the Proxy API.

Function mocks

The ability to create a mock of a standalone function.
Some libraries may be limited to working with objects and methods of objects, but not functions which are not part of an object.

Undefined behavior

Undefined behavior happens when the runtime behavior of a type breaks the guarantees offered by its type. Then the outcome of program execution is undefined (aka. "random").
A mocking framework exhibits undefined behavior when it fills in unspecified method calls or member accesses with a default return value which does not match the type of the method or member (most often undefined or null). A well designed mocking framework must throw an exception or fail the test whenever behavior for a given action has not been specified, and it is not possible to (type-)safely fallback to a default behavior.

Return value

Mocking the value returned by a function or getter.

Mocking an exception

Mocking a thrown exception upon invocation of the method or getter. Note that if the library supports fakes, then to mock an exception you just create a fake which throws that exception. No big deal.

Mocking a getters or members

Mocking the value of a member of an object or class, or the return value of a getter (these are equivalent in JavaScript).

Call forwarding

Ability to delegate a call or member access to the backing instance of the mock.

Call fake

Defining a custom function to call when the mocked method is called or the mocked member is accessed.

Deep mocking

Ability to fake the result of a chain of calls or member accesses.

Unexpected call verification

Verifying that no unexpected calls occurred. Most libraries throw an exception if an unexpected call occurs.
safe-mock returns an object which throws an exception when it is used. This unusual behaviour was most likely chose in order to preserve the merged setter/mock design.
A library which silently returns an undefined value upon unexpected calls violates type safety and fails this criterion.

Missing call verification

Ability to verify that the method was called a given number of times.

Matchers

Argument matchers lets you define different expectations for different invocations of a method based on a set of acceptable parameter values. Similarly, matchers can be used to verify different calls to a method.

Verifying setters

Ability to verify that a member of a class or object has been set during the test execution.

Return value type checking

Type check the return value provided to a mock.

Arguments type checking

Checks the type of the arguments passed to the mock call and received by a fake. This includes the type-safety of argument matchers.

Templated functions

Preserve the type safety of functions which take type arguments. For instance:

interface MyVault {
    getSecretRecipe<T extends string>(s: T): T extends '5cr37' ? string : number;
}

const mock = createMock<MyVault>();

mock.getSecretReceipe('aaa'); // number

// Some frameworks may think the return value here is `number`, which is wrong.
mock.getSecretReceipe('5cr37'); // string

Mock is the stub

Whether the object you use to set expectations is the same object you pass to the tested code.
There are conceptually two objects for each mock you create: One is a handler you use to describe the behaviour of the mock, the other is the actual object used by the code you want to test.

Some frameworks chose to merge these two entities into a single object. The result is a lighter syntax and less error prone library because you can't mistake the expectation setter with the mock. On the other hand, using the same symbol for two purposes means it is always ambiguous whether the object is currently being used to set an expectation or by the tested code.
This ambiguity can lead to confusing type or runtime errors depending on how the framework is implemented. It also makes some features harder or impossible to implement.

Frameworks which chose to keep these two functionalities separated are more verbose and more confusing to beginners. However, they generally contain less black magic which makes them more predictable and less prone to edge cases.

Error reporting

A good mocking framework produces descriptive error messages to help the developer. This criteria is one of the most important. Unfortunately, an objective comparison of the quality of error reporting between the different frameworks would require a lot of time and energy. Therefore, this criteria has not been evaluated, but it is shown in the grid to remind the reader of the importance of error reporting. Anyone willing to contribute a review of error reporting in these frameworks would be very valuable.

Other libraries

The following are additional mocking libraries not included in the main comparison. Any feedback would be appreciated.

  • ts-mock-imports : Seems to focus on patching imported modules.
  • ts-auto-mock : Automatically generates mock objects from a given type, using a TypeScript transformer.
  • ts-mocks : Specific to Jasmine.
  • strong-mock : States some limitations on getter and call forwarding
  • typemoq
Clone this wiki locally