[RFC] Typesafe module mocking based on package.json standards

[kasperpeulen]

Status: implemented

🖼️ Overview

Outcome

We want users to be able to change (mock out) the implementation of a module when it runs inside of storybook.

Purpose

1. Some users prefer mocking out modules over a tool like MSW or MirageJS, for various reasons. Some people are used to module mocking because of the flexibility that tools such as Jest and Vitest allow here. Some people prefer writing mock for their internal models, as writing mocks for external http-data is sometimes harder to write or harder to maintain.
2. With RSC, users can write React components that touch files that can not even run in the browser. For example, you might want to mock out your data access layer which uses DB calls under the hood. Or a file that uses other node API's such as fs and path.
3. There are other non-http related modules you might want to mock out or spy on for various reasons in storybook.

💡 Proposed Solution

The proposal is to use package.json imports as a standard based way to mock out modules in storybook. We feel that the time is right to at least giving module mocking without any magic a fair try, before we consider other, more magical, approaches.

The way that Jest (later adopted by Vitest) solves module mocking, evolved in a time where no ESM standards existed at all. The API used here requires very little boilerplate, which is nice, but the amount of magic needed also has some drawbacks:

• The test file is no standard JS anymore. For example jest.mock/vi.mock statements are rewritten to be hoisted at the top and can not easily consume variables declared in the file.
• TS doesn't really know which files are mocked or not. So you have to forcefully tell TS, that it can safely assume that a function has a Mock signature.

Those problems don't exist in a fully explicit standard based mocking solution, which gives us extra motivation to explore this angle.

Package.json imports

The "imports" field in package.json allows you to configure import aliases that only apply to imports from within the package itself.

• https://nodejs.org/api/packages.html#imports
• https://nodejs.org/api/packages.html#subpath-imports

// package.json
{
  "imports": {
    "#dep": {
      "node": "dep-node-native",
      "default": "./dep-polyfill.js"
    }
  },
  "dependencies": {
    "dep-node-native": "^1.0.0"
  }
} 

Entries in the "imports" field must always start with # to ensure they are disambiguated from external package specifiers.

Absolute imports

This feature allows for writing absolute imports in Node natively. There is no bundler or compiler magic needed to get it to work. Just add this to your package.json:

{
  "imports": {
    "#*": "./*"
  }
}

And now you can import the path:

• {root}/components/foo.js
• with import { foo } from '#components/foo'
• or with import { foo } from '#components/foo.js' if you use ESM natively in Node.

Other absolute import conventions have been popularised such as importing @/components/foo and manually configuring this in TS and in your bundler config. However, now TS 5.4 supports this package.json feature and has autocompletion support for it, it seems that Next.js wants to move to package.json imports as well for absolute import support:
https://x.com/sebmarkbage/status/1765828741500981475?s=20

Kennt C. Dodds popularized absolute imports like this in the Remix community already for some time:
https://github.com/epicweb-dev/epic-stack/blob/main/docs/decisions/031-imports.md

Potentially (and hopefully), all frameworks/stacks are gonna gradually converge to this native way of absolute imports.

Conditional absolute imports

One exciting opportunity for Storybook here is that package.json imports can be made conditional in the same way as package.json exports:

{
  "exports": {
    "import": "./index-module.js",
    "require": "./index-require.cjs"
  },
  "type": "module"
}

https://nodejs.org/api/packages.html#conditional-exports

The built-in Node conditions are "import", "require", "node", "node-addons" and "default" , however, any other string can be used and triggered by running node with -C options:

node -C storybook ./foo.js

Will resolve the conditional import to storybook entry, if it exists.

Other commonly accepted package conditions (for example, implemented by bundlers) are:

• "types" - can be used by typing systems to resolve the typing file for the given export.
• "browser" - any web browser environment.
• "development" - can be used to define a development-only environment entry point, for example to provide additional debugging context such as better error messages when running in a development mode.
• "production" - can be used to define a production environment entry point.

Other other runtimes, platform-specific condition are:

• deno, react-native, netlify, electron, bun, react-server, fastly

To name a few.

Implement storybook conditions to our builders

Storybook of course doesn’t run on node, but the web (and its tooling) has embraced package.json conventions as the way to resolve imports. it would be natural to implement module mocking in a way that is officially supported by package.json.

For example, if the users add the following to package.json:

{
  "imports": {
    "#api/todo": { "storybook": "./api/todo.mock", "default": "./api/todo" },
    "#*": "./*"
  },
}

Would mean that whenever the user imports:
import { getAllTodos } from '#api/todo'
It will resolve when running in storybook to:
{root}/api/todo.mock.ts if that file exists and otherwise fallback {root}/api/todo.ts
It is quite unfortunate for our use case that the "spec" doesn't allow for a one-liner like this:

{
  "imports": {
    "#*": {
      "storybook": ["./*.mock", "./*"],
      "default": "./*"
    }
  }
}

Actually webpack and TS do allow this, but they implement the spec "incorrectly".

Node does allow a fallback array, when validation fails, but it won't check if the file exists, as that would be too slow. All validation rules are implemented statically, so that it can be super fast.

Maybe at some point, we should make a proposal to change this spec with the module mocking use case in mind.
My feeling is that performance is actually not that big of issue if the validation array is used for conditions that only resolve for development or testing use cases. Even more so as often a bundler is used in those cases. And for the web this is almost exclusively true, as even HTTP/2 is not fast enough to outcompete a bundler.

Possibly the spec should be expanded with that in mind? Maybe node could allow a feature flag for those use cases? A flag that test runners could apply etc.

Advantages of using a standard

This feature allows use to module mocking in storybook completely based on standards.

The advantage is that by using the "imports" field, we don’t have to do anything special for vite or webpack , they just understand the syntax.

Similarly, Eslint or TypeScript doesn’t have to configured specially to make sure they understand this syntax. Since TS 5.4, package.json imports are also auto-completed!
https://devblogs.microsoft.com/typescript/announcing-typescript-5-4/#auto-import-support-for-subpath-imports

Comparison with vitest/jest manual mocking

This way of module mocking is comparable with how vitest and jest look for mock files in a adjacent __mocks__ directory:
https://jestjs.io/docs/manual-mocks

In vitest/jest, when you want to use the mock instead of the real implementation, you still need to explicitly call jest.mock('./moduleName') or vi.mock('./moduleName') in your test file.

With package.json imports, the mocked module will always be used when running storybook. To make sure the user can opt-out of mocking the module, we can advice mock file content similar as this:

// ./services/api/todos.mock.ts
import { fn } from '@storybook/test';
// Note that we don't need to build an "import actual" convention.
// as the user can use relative imports, to get the actual implementation
import * as todos from './todos'; 

export const getAllTodos = fn(todos.getAllTodos);
export const getTodoById = fn(todos.getTodoById);

This means that by default the unmocked version will be used in storybook as well, it will only be spied on by default, so that you can assert on it in your play function:

import * as todos from '#services/api/todos.mock';

const Default = {
  play: async() => {
    ...
    todos.getAllTodos.mock.calls > 0 // etc.
  }),
}

When you want to change the mock implementation for a certain test, you can use:

import * as todos from '#services/api/todos.mock';

const Default = {
  beforeRender() { // RFC for beforeRender is coming!
    todos.getAllTodos.mockResolvedValue(someTodos);
  },
}

Before each story loads, we restore all mocks to the default implementation (we already do this, for spies attached to args), so that the original function is restored.

If you do want a default mock implementation for all stories you can do so in your preview:

// .storybook/preview.ts
const preview = {
  beforeRender() {
    todos.getAllTodos.mockImplementation(() => someTodos);
  }
}

One other advantage of defining your mock implementation in preview/meta or the story itself, is that you have access to the story args:

// .storybook/preview.ts
const preview = {
  beforeRender({args}) {
    todos.getAllTodos.mockImplementation(() => args.$todos);
  }
}

Which would integrate nicely with another proposal (RFC coming) where $-prefixed-args are treated as non-component args. So won't be automatically applied to the component.

Automatic boilerplate generation

With this pattern, the actual api/todos.mock.ts file can be code-generated automatically. ES exports needs to be statically defined. But because they are statically defined we can quite easily with AST parsing find out the exports of the module “to-mock”. So that we can code generate files like these:

// ./api/todos.mock.ts
import { mock } from '@storybook/test';
import * as mod from './todos';

const mocked = mock(mod);

export const { getAllTodos, getTodoById, getFilteredTodos } = mocked;
export default mocked.default;

One advantage of code-generation over the more dynamic approach that vitest and jest use, is that TS will work out of the box.

The pattern the user would need is always importing the actual “mocked” file in their story file:

import { getAllTodos } from '#services/api/todos.mock'; // importing todos.mock, not todos, to make sure TS understand the type
import { mocks } from '@storybook/test';
const Default = {
  beforeRender() {
    // TS knows that this is a function has a `mockImplementation` method
    getAllTodos.mockImplementation(() => someTodos);
  }
}

In some sense, this code generation, makes sure that the story file is a bit cleaner as if we would adopt a more magical approach such as vitest does:

import { vi } from 'vitest';
import * as todosModule from '#services/api/todos';

vi.mock('#services/api/todos');
const todos = vi.mocked(todoModules);

const Default = {
  beforeRender() {
    // TS knows that this is a function has a `mockImplementation` method
    todos.getAllTodos.mockImplementation(() => someTodos);
  }
}

Of course, Vitest approach has also a lot of advantages, but I’m eager to at least start with a zero magic approach, just using standards, and iterate on that to make that as easy as possible to write.

What to do with mocking external modules?

There are a couple of options:

1. Making default browser mocks for all many built-in node packages. This is not as hard as it sounds, as there is much prior art here. So people don’t need to mock out node:fs etc, it will be written to a memory “fs”. We should take performance in mind here though, and only do this optionally.
2. In the next framework we can automatically mock some files that are using AsyncLocalStorage under the hood (such as next/headers)
3. Advise people to always wrap their dependency in their own module. I think this is often the right thing to do. Instantiate prisma in a utils/db.ts and mock that file out, instead of prisma
4. If the option 2 feels a bit weird, you could also mock an external module such as bcryptjs by changing it to an #bcryptjs import that is made conditional in package.json. This pattern forces you though to write this external dep everywhere you use it as #bcryptjs.
5. People can always add some custom vite or webpack aliases, as a last resort.

[chakAs3]

I am impressed by the effort put into this RFC. I'd like to echo @kasperpeulen's sentiment and contribute some best practices from Nuxt for module authoring, particularly regarding auto-import facilitation or aliasing for vite/webpack imports, which align closely with his objectives.

Best Practices

• Prefer named exports rather than default export. This helps reduce CJS conflicts. (see Default exports section)

• Avoid depending on Node.js built-ins and CommonJS or Node.js-only dependencies as much as possible to make your library usable in Browsers and Edge Workers without needing extra polyfills.

• Use new exports field with conditional exports. (read more).

{
  "exports": {
    ".": {
      "import": "./dist/mymodule.mjs"
    }
  }
}

[musjj]

I really LOVE this particular approach of vitest:

as if we would adopt a more magical approach such as vitest does:

import { vi } from 'vitest';
import * as todosModule from '#services/api/todos';

vi.mock('#services/api/todos');
const todos = vi.mocked(todoModules);

const Default = {
  beforeRender() {
    // TS knows that this is a function has a `mockImplementation` method
    todos.getAllTodos.mockImplementation(() => someTodos);
  }
}

So PLEASE let this be possible with Storybook too. The nice thing about it is that it keeps everything related to the story local inside the story file itself, instead of having to litter your project and exports with mock modules.

Writing separate modules for everything we want to mock is just so painful. Code generation might make this easier, but it pushes an extra build step complexity and yet another potential failure point to the user, so I honestly don't feel excited about it. It also doesn't solve the issue of having our project be polluted by mock modules.

[kasperpeulen]

We are considering it for vite-based projects. As vitest has a package for that now:
https://github.com/vitest-dev/vitest/blob/main/packages/mocker/EXPORTS.md

The nice thing about it is that it keeps everything related to the story local inside the story file itself, instead of having to litter your project and exports with mock modules.

I wonder, though, how often you want to use module mocking in a storybook project. And for which use cases? Do you prefer module mocking over something like msw?

[musjj]

We are considering it for vite-based projects.

Thanks, that's really cool! Would it be possible to have it for Webpack/Next.js-based projects too?

Do you prefer module mocking over something like msw?

MSW is good for mocking network calls, but on Next.js I frequently import and call functions that can only run on the server (e.g. DB client) within server actions. MSW can't exactly help with that.

On the mean time, I've been using this addon: https://github.com/ReactLibraries/storybook-addon-module-mock, and it's working pretty well so far! But still, an official solution would be nice.