Rstest Roadmap

[9aoy]

Rstest now supports core testing scenarios including Node / DOM testing, mocking, multi-project testing, and coverage collection, and has been successfully implemented in Rstack projects (including Rsbuild, Rslib, Rspack, and others).

The project is currently in active development, and we will continue to refine functionality and enhance the user experience based on community feedback.

Rstest features track

Basic Features

• Expect API (Jest Compatible)
• Lifecycle hooks (beforeAll、beforeEach、afterAll, etc.)
• Setup files
• Snapshot testing (Jest Compatible)
• Source map
• Test API (describe、test、only、each, etc.)
• In-source testing
• Default Reporter
• Out-of-the-box TypeScript / JSX support
• CLI
  • rstest run
  • rstest watch
  • rstest list
• Spy & Rstest utility (rstest.spyOn、rstest.fn, etc.)
• Mocking
  • mock
  • doMock
  • unmock
  • importMock
  • importActual
  • resetModules
  • hoist
• Config
  • Test configuration support, such as include, pool, setupFiles, etc.
  • Build configuration support, such as resolve, transform, external, etc (based on rsbuild).
• API Documentation
• Auto Mock (rs.mock('./m.ts', { mock: true }))
• Pool & isolated
  • forks
  • vmForks
  • threads
  • vmThreads
• Environment
  • node
  • jsdom
  • happy-dom
  • custom environment 🚧
• ESM support
• Watch mode
• CLI shortcuts
• Coverage
  • v8
  • istanbul
• Integration
  • Out-of-the-box Rsbuild testing support
  • Out-of-the-box Rslib testing support
  • Out-of-the-box Rspack testing support

Advanced Features

• Projects
• Out-of-the-box Lynx testing support
• Rsdoctor integration
• More builtin reporters & Stablize Reporter API
  • HTML Reporter
  • JSON Reporter
  • GitHub Actions Reporter
  • JUnit Reporter
  • Custom Reporter
• Rstest UI (Provide UI to view and interact with tests)
• VS Code extension
• Test Sharding & merge reports
• Support filter tests via related source files
• Browser Mode

Further Features

• Benchmark testing
• Type testing
• AI Exploration

[Ghostblad3]

Hello there. One of the biggest issues of Jest is that it reads all the necessary modules for each test but does not cache them during runtime.

In front-end development most of these modules are the same. So you end up reading for each test thousands of files again and again.

A simple React component without 3rd party libs, need around 800 module files. However a React component that imports a component library, and icon library, a state management library etc can easily peak at more than 5k module files during tests because a lot of these 3rd party libraries are barreled. Now imagine you if you got 200 different React components where each needs about 5k files and you want to run all the tests you need to hit the disk 200 * 5_000 = 1_000_000 times.

On a decent personal laptop that's not really a problem.

However in a restricted enterprise environment where you have a cheap enterprise U series low powered laptop and you are forced to use windows with limited permissions, have bit-locker enabled which degrades SSD performance, security software monitoring the background and an aggressive windows defender that thinks that the node process is a ransomware because it reads thousands of files really fast it gets unbearable. Note that even requesting access for WSL can be denied since you aren't messing with docker and microsevices if your role is front-end dev only and having a folder scan exclusion and node process scan exclusion from the windows defender is a big no from the IT security guys for security reasons (I don't disagree with the window defender part).

I started experimenting with modifying Jest source code and added high precision timers in the function that did readFileSync from the disk. I observed that reading for example 5k node module files alone could take 135 secs which is insane to me and can lead to execution times of 3 minutes for a single test file since jest loads reads other files from the file system before that like jsdom related files, package json files and also it needs to spin up an isolated VM for running the test etc.

1. My proposal is to cache and share the node modules during runtime. This can avoid I/O bottleneck especially in the type of environment I described above.

2. My second proposal is a bit crazy but it can dramatically improve performance in I/O starved environments.
   Create a disk based module pool. Instead of reading a module file at a time, add it after the tests finish to a file pool in the disk.

module-pool.json

[
  {
    "file_path": "C:\\...\\file.js",
    "content": "<<content_of_file>>",
    "last_modified": "<<time_stamp>>",
    "last_accessed": "<<time_stamp>>"
  },
  {
    "file_path": "C:\\...\\file2.js",
    "content": "<<content_of_file>>",
    "last_modified": "<<time_stamp>>",
    "last_accessed": "<<time_stamp>>"
  }
  // ... more entries
]

The last_modified contains the last time the file was modified in the file system.
The last_accessed contains the last time a test needed the specific module.

The last_modified can help us invalidate the content of the file if it was updated and the last_accessed can help us determine if the file has to be requested for a long time (user could provide an option to discard modules from the pool if they haven't been used for example 2 months during the test runs).

Since CI/CD can be tricky to determine last_modified (node_modules dependencies are reinstalled on each CI/CD unless you can retain somehow the old modules after each pipeline execution, I am not sure if it is worth it and what kind of ) this second proposal could be under a config value like "localDevelopmentEnvironmentModulePool": true/false and only be used during local development.

I am looking forward to hear your feedback.

[9aoy]

@Ghostblad3 Thank you for your suggestion. Currently, when rstest executes a test file, it has module caching to prevent redundant file loading.
However, between different test files, there is no caching for module loads due to test isolation. Fortunately, if a package is not marked as external, it will be bundled by rspack (also known as transform), and its file content will be loaded into memory, preventing redundant disk reads.

As for long-term caching optimizations, rspack's Persistent Cache may be supported in rstest in the future.

[Ghostblad3]

@9aoy Thank you! Could you elaborate why modules should not be shared between tests due to isolation? Modules are static, they never change. If X components uses a Material UI button I don't see a problem for the modules required by the button to be shared between the tests. Sharing them dramatically improved performance.

[9aoy]

@Ghostblad3 Rstest runs every test file isolated by default. If your project has no side effects, you can set isolate: false to turn off isolation to optimize performance (There will be some errors that do not meet expectations, and we will fix them ASAP).

[Ghostblad3]

@9aoy Does isolate: true will cache all modules loaded for each test so that they are not re-read from disk after a test finish and another one start?

[9aoy]

Does isolate: true will cache all modules loaded for each test so that they are not re-read from disk after a test finish and another one start?

When isolate: false is enabled and tests are within the same pool, module loading will be cached.

[derekdon]

Loving the suite.

Assuming the vi.mock type apis haven't been implemented yet right? ie. no matter what variants I try, they take no effect...

rstest.mock("../src/import/remote", () => ({
    getRemoteService: () => mockRemoteService,
}));
...
const module = { someExport: "value" };
rstest.mock("@module-federation/enhanced/runtime", () => ({
    loadRemote: rstest.fn().mockResolvedValue(module),
}));

I just noticed a branch called fix_mock, with plugin tweaks... might explain it.

[fi3ework]

I just noticed a branch called fix_mock, with plugin tweaks... might explain it.

Oh, rstest as a alias of rs (or vice versa), is not merged into latest version now. You could use rs as of now.

[travzhang]

Hi, I'm the author of CanyonJS, an end-to-end coverage collection platform designed for frameworks like Playwright. It works by using babel-plugin-istanbul to instrument code at the AST level and collects coverage data at runtime.

Internally, we've extended this plugin to support advanced features like hit/map separation, allowing us to reduce bundle size and make coverage collection more production-friendly.

I'd love to contribute to the rstest project, specifically around its coverage functionality. I'm particularly interested in building a custom coverage provider similar to @rstest/coverage-istanbul or @rstest/coverage-v8, with support for code instrumentation and coverage report generation during test execution.

I'm currently working on a minimal runnable demo and plan to share it before July 25.

Please let me know if you're open to contributions in this area—I'd be happy to help push coverage support forward in rstest. Thanks!

[fi3ework]

@travzhang Thanks for the sharing, Rstest's coverage ability is still under development, and the provider mechanism is not implemented as of now. We had planned to implementation it within this month or next.

Implementation coverage core functionality will be a rather bigger task, if you're interested in implementing it, please come up a RFC or issue to demonstrate the API design before start development.

Implementation CanyonJS' plugin after the coverage provider system implementing is sure feasible and welcomed. We'd love to help if you encounter any issue.

By the way, I'm interested in how you will integrate CanyouJS with Rstest? I only saw the recipes with Next/Vite/Lynx/RN, which are all app frameworks instead of testing framework.

[travzhang]

#398