Allow ESM stubbing for functions in Vite

[adamdehaven]

What would you like?

Utilizing Vite + Vue and Cypress Component Test Runner, how would you stub a composable function since you can't stub the default export?

I can't find a decent example that doesn't utilize Babel, and the only solution we have come up with is exporting an object with methods that can be stubbed, which, to be honest, would be a large refactor.

When stubbing the default export, as shown below, an error is thrown: ESModules cannot be stubbed. I know this is a valid error (here's a great write-up for reference); however, we need the ability to stub the default exports of imported modules.

// Composable function
import { ref } from 'vue'

export default function useToggle (initialValue = false) {
  const enabled = ref(initialValue)

  return { 
    enabled
  }
}

// Component usage
import useToggle from '../composables/useToggle'
.... 
setup(props) {
  const { enabled } = useToggle(false)

  onMounted(() => console.log(enabled.value)
}

// test.spec.ts
import { ref } from 'vue'
import useToggle from '../composables/useToggle'

// This doesn't work, and we're not using Babel since switching to Vite
// Throws an error: `ESModules cannot be stubbed`
cy.stub(useToggle, 'default').returns(
  { 
    enabled: ref(true),
  }
)

Why is this needed?

It's a standard in Vue 3 to move stateful logic into composable functions. Composables typically export a default function (not an object) and devs need the ability to stub the outputs of a composable.

There is currently a workaround; however, it would require refactoring large blocks of code within our (and most) application whereby the composable exports a utility function that is used to set the returned values. This workaround (shown below) is a bit cumbersome and requires a very explicit, non-standard way of writing composable functions.

Here is the same composable provided above, rewritten for the workaround (not ideal):

// Composable function
import { ref, Ref } from 'vue'

export const getToggleData = {
  enabled: (initialValue = false): Ref<boolean> => ref(initialValue)
}

export function useToggle(initialValue = false) {
  const enabled = getToggleData.enabled(initialValue)

  return {
    enabled,
  }
}

// Component usage
import useToggle from '../composables/useToggle'
.... 
setup(props) {
  const { enabled } = useToggle(false)

  onMounted(() => console.log(enabled))
}

// test.spec.ts
import { ref } from 'vue'
import { getToggleData } from '../composables/useToggle'

// This works, but isn't ideal due to the way the composable has to be written (not the norm)
cy.stub(getToggleData, 'enabled').returns(true)

Other

No response

[adamdehaven]

@JessicaSachs @lmiller1990 let me know if I can provide any additional info

[JessicaSachs]

Thanks for opening this issue. We should be doing what the Vitest plugin does to enable ESM mocking.

[lmiller1990]

The Vitest code doesn't look too bad. It might be a good solution for now (to help vite-dev-server) users. Long term, I wonder what's the best option for people who (eventually) want to write e2e tests using native ESM modules, or another dev server?

A dev-server specific mock/stub doesn't seem ideal, although happy to look into this in the short term until we better know how to handle the general problem of ESM module mock/stubs.

We will discuss internally and figure out the logistics/prioritization of this!

[FelipeLahti]

Can we include allowing esm stubbing functions in NextJS as well?
Is that on the roadmap? @JessicaSachs

That's the only thing preventing us to use the new SWC compiler on NextJS. Currently, we're using the following .babelrc to make stubbing work.

{
  "presets": ["next/babel"],
  "plugins": ["@babel/plugin-transform-modules-commonjs", "babel-plugin-styled-components"]
}

[JessicaSachs]

👋 You want @baus or @ZachJW34 for Next-y or roadmap planning things.

[illegalnumbers]

Is there any update for the latest version of Cypress? When trying something similar with a composable I get the error Cannot stub non-existent own property default rather than the ESModules cannot be stubbed. When trying to stub out apply or call it seems like the function doesn't actually get replaced in the component.

[adamdehaven]

@illegalnumbers as a workaround, I export the composables from another file, then stub.

// composables/index.ts
import useOrg from './useOrg'

export {
  useOrg
}

// Component file (usage example)
import composables from '../composables'

const org = composables.useOrg()

// Test spec file
import composables from '../composables'

cy.stub(composables, 'useOrg').returns({ data: [] })

[lmiller1990]

Sorry fam, we did not work on ESM stubbing yet. I'd really like to see this implemented, I hope we can look at it soon.

Vitest supports this but they run everything in a Node.js context. I think the first step is looking at what they do and finding out if we can do the same thing.

Is there any other runners supporting ESM stubs? Jest now has ESM support natively - I wonder how they implement it? I'm thinking the most likely rely on the Node.js module resolution, too - I don't know of any browser runners that support ESM and stubs.

If anyone wants to explore this, I can definitely help out or make some suggestions. Especially if someone can find a reference implementation, I can assist with the "here is where you need to put the code in Cypress".

[iambumblehead]

I'm interested in adding browser support to esmock. it would be awesome if someone would add a cypress test folder alongside esmock's other test folders, and inside the test folder, add a passing test and a broken/failing test that uses esmock in the right place to try and browser-import a module with mock import tree.

[lmiller1990]

Oh nice! Hadn't seen this project, this looks promising.

[lmiller1990]

I think it's time to tackle this, going to get some resources into this in our next sprint. Not sure on the complexity or how best to solve this, but we can at least start investigating.

[lmiller1990]

Hey team! Please add your planning poker estimate with Zenhub @astone123 @marktnoonan @mike-plummer @warrensplayer @ZachJW34

[muratkeremozcan]

There is a nice repo we can test this on once delivered: https://github.com/muratkeremozcan/tour-of-heroes-react-vite-cypress-ts. Enable the lines with the comment // TODO: wait for https://github.com/cypress-io/cypress/issues/22355

[lmiller1990]

Hey team! Please add your planning poker estimate with Zenhub @astone123 @marktnoonan @mike-plummer @warrensplayer @jordanpowell88

[mike-plummer]

Update on ESM stubbing

Unfortunately this is a non-trivial problem due to how ES Modules are designed. The ESM spec requires modules to have an immutable namespace, and since Cypress component tests run inside the browser and use the browser's module loader to ensure spec compliance it isn't possible to do something like this:

import MyModule from './module.js'

// `MyModule` is a sealed namespace, can't resassign/add/delete direct members
MyModule.something = 'somethingElse'

Other testing tools like Jest & Vitest work around this in a couple different ways:

1. Replacing the Node module loader so that modules are mutable
   This nicely sidesteps the issue for tools that run tests within Node, but also means your code isn't spec-compliant when it runs. This could disguise actual problems in your tests.

2. Mocking an entire module for the duration of a spec
   Cypress APIs are designed to allow ad-hoc stubbing/spying throughout a test (see cy.stub and cy.spy), and we would like to maintain the capability. Replacing an entire module for the whole spec is a very different way of structuring tests and can be a bit restrictive depending on your application structure and testing use case.

What are your options today?

Rest assured, we are working on a way to support this in a way that doesn't have a massive breaking change and doesn't change the way your code runs. In the interim, there are a couple workarounds that I'll try to outline here:

1. Export wrappers to sidestep module immutability
   As stated above, this doesn't work:

   /// module.js
   export function myFunc() {}
   
   // module.spec.js
   import * as MyModule from './module.js'
   cy.stub(MyModule, 'myFunc')

   However, this does work:

   /// module.js
   function myFunc() {}
   export const MyModule = {
       myFunc
   }
   
   // module.spec.js
   import { MyModule } from './module.js'
   cy.stub(MyModule, 'myFunc')

   This is because the module namespace is immutable, but the module can export members that are mutable. In this case, the wrapper object MyModule is mutable, allowing us to stub anything within it. This requires you to structure and use your modules in a particular way and it is only an option for modules under your control.

2. Use an importmap
   If you have modules that you always want mocked out and are using a fairly modern browser you can add an importmap to your component-index.html to tell the browser to resolve a custom implementation anytime it is requested. Note that this will impact all uses of the module throughout your test suite, but it is an easy way to replace modules you don't want running in your tests or that you want to behave slightly different in all tests.

   Note that the "mock" implementation will itself be loaded by the browser as an ES Module, so it will be subject to the same restrictions as the original. This means you can't use cy.stub on any namespace member of the "mock" implementation, but you can write any logic you want into the mock to be shared across all of your tests.

3. Avoid stubs & spies
   Stubbing and spying can be an anti-pattern in testing depending on how they're used. You might consider whether it's possible to refactor a component to split out behavior you want to stub/spy into a separate component/hook/etc.

Is Cypress considering any changes?

Short answer - we aren't sure yet. We're kicking around a few ideas, and if you know of another we'd love to hear about it.

To get the same sort of behavior as Jest and others we could introduce a new API like cy.mock('./module.js', {... }) to mock a module for the entirety of a test. This would be a bit less capable than our existing APIs and could potentially be a breaking change. There are also potential issues around how this could work with things like custom commands since the mocking would have to occur before anything accesses that module. Finally, this sort of API would really just be a wrapper around the importmap idea above so you already have the capability to do it, just with a bit more configuration.

Another option we're considering is a plugin that rewrites ES Modules as they're served by your dev server so that they're mutable. This is a fairly complex thing to get right, and we aren't 100% sold on this being the right thing to do. One of the major principles of Cypress is that we try to be as standards-compliant as possible so we don't accidentally hide problems in your code. We'd love to hear your thoughts on this approach, and if anyone knows of a tool that already does this (or is interested in writing one) let us know!

[lmiller1990]

There are still some bugs we are working on fixing, please share a repro of any you run into. Latest would be @lmiller1990/vite-plugin-cypress-esm.

[lmiller1990]

This is working pretty well, a bunch of fairly complex React component tests are passing: muratkeremozcan/tour-of-heroes-react-vite-cypress-ts#103

This (as a npm module) will be live soon under @cypress/vite-plugin-cypress-esm (name TBA). It's still got work, but this will let us get more feedback - the more feedback and testing we can get from real projects, the sooner we can make it part of the core offering.

[marktnoonan]

This is published now: https://www.npmjs.com/package/@cypress/vite-plugin-cypress-esm

[lmiller1990]

I found this is not working for some basic cases: https://github.com/lmiller1990/esm-bug

I will file an issue.

[dwilt]

@marktnoonan how would I go about using mocking ESM modules with a NextJS project?

[lmiller1990]

@dwilt you could try: https://www.npmjs.com/package/@cypress/vite-plugin-cypress-esm

Are you actually using ESM (eg "type": "module")? Or just the import/export syntax (depending on your config, that may be transpiled to CJS).

If you have a specific reproduction of something not working, happy to take a look.

[andyhqtran]

@dwilt you could try: npmjs.com/package/@cypress/vite-plugin-cypress-esm

Are you actually using ESM (eg "type": "module")? Or just the import/export syntax (depending on your config, that may be transpiled to CJS).

If you have a specific reproduction of something not working, happy to take a look.

@lmiller1990 Trying out the plugin returns this error:

We just use the import/export syntax. We don't have "type": "module" set in our package.json since we get this error:

core:test:cy: (node:24174) ExperimentalWarning: Custom ESM Loaders is an experimental feature and might change at any time
core:test:cy: (Use `node --trace-warnings ...` to show where the warning was created)
core:test:cy: (node:24174) ExperimentalWarning: The Node.js specifier resolution flag is experimental. It could change or be removed at any time.
core:test:cy: /Users/use/Library/Caches/Cypress/12.17.3/Cypress.app/Contents/Resources/app/packages/telemetry/dist/span-exporters/ipc-span-exporter.js:11
core:test:cy:         super({});
core:test:cy:         ^
core:test:cy: TypeError: Class constructor OTLPTraceExporter cannot be invoked without 'new'
core:test:cy:     at new OTLPTraceExporter (/Users/use/Library/Caches/Cypress/12.17.3/Cypress.app/Contents/Resources/app/packages/telemetry/dist/span-exporters/ipc-span-exporter.js:11:9)
core:test:cy:     at Object.<anonymous> (/Users/user/Library/Caches/Cypress/12.17.3/Cypress.app/Contents/Resources/app/packages/server/lib/plugins/child/require_async_child.js:9:18)
core:test:cy:     at Module._compile (node:internal/modules/cjs/loader:1254:14)
core:test:cy:     at Module.m._compile (/Users/use/Library/Caches/Cypress/12.17.3/Cypress.app/Contents/Resources/app/node_modules/ts-node/dist/index.js:857:29)
core:test:cy:     at Module._extensions..js (node:internal/modules/cjs/loader:1308:10)
core:test:cy:     at Object.require.extensions.<computed> [as .js] (/Users/use/Library/Caches/Cypress/12.17.3/Cypress.app/Contents/Resources/app/node_modules/ts-node/dist/index.js:859:16)
core:test:cy:     at Module.load (node:internal/modules/cjs/loader:1117:32)
core:test:cy:     at Function.Module._load (node:internal/modules/cjs/loader:958:12)
core:test:cy:     at ModuleWrap.<anonymous> (node:internal/modules/esm/translators:169:29)
core:test:cy:     at ModuleJob.run (node:internal/modules/esm/module_job:194:25)```

[lmiller1990]

Will need a minimal reproduction to give any useful advice!

[AtofStryker]

@dwilt you could try: npmjs.com/package/@cypress/vite-plugin-cypress-esm
Are you actually using ESM (eg "type": "module")? Or just the import/export syntax (depending on your config, that may be transpiled to CJS).
If you have a specific reproduction of something not working, happy to take a look.

@lmiller1990 Trying out the plugin returns this error:

We just use the import/export syntax. We don't have "type": "module" set in our package.json since we get this error:

core:test:cy: (node:24174) ExperimentalWarning: Custom ESM Loaders is an experimental feature and might change at any time
core:test:cy: (Use `node --trace-warnings ...` to show where the warning was created)
core:test:cy: (node:24174) ExperimentalWarning: The Node.js specifier resolution flag is experimental. It could change or be removed at any time.
core:test:cy: /Users/use/Library/Caches/Cypress/12.17.3/Cypress.app/Contents/Resources/app/packages/telemetry/dist/span-exporters/ipc-span-exporter.js:11
core:test:cy:         super({});
core:test:cy:         ^
core:test:cy: TypeError: Class constructor OTLPTraceExporter cannot be invoked without 'new'
core:test:cy:     at new OTLPTraceExporter (/Users/use/Library/Caches/Cypress/12.17.3/Cypress.app/Contents/Resources/app/packages/telemetry/dist/span-exporters/ipc-span-exporter.js:11:9)
core:test:cy:     at Object.<anonymous> (/Users/user/Library/Caches/Cypress/12.17.3/Cypress.app/Contents/Resources/app/packages/server/lib/plugins/child/require_async_child.js:9:18)
core:test:cy:     at Module._compile (node:internal/modules/cjs/loader:1254:14)
core:test:cy:     at Module.m._compile (/Users/use/Library/Caches/Cypress/12.17.3/Cypress.app/Contents/Resources/app/node_modules/ts-node/dist/index.js:857:29)
core:test:cy:     at Module._extensions..js (node:internal/modules/cjs/loader:1308:10)
core:test:cy:     at Object.require.extensions.<computed> [as .js] (/Users/use/Library/Caches/Cypress/12.17.3/Cypress.app/Contents/Resources/app/node_modules/ts-node/dist/index.js:859:16)
core:test:cy:     at Module.load (node:internal/modules/cjs/loader:1117:32)
core:test:cy:     at Function.Module._load (node:internal/modules/cjs/loader:958:12)
core:test:cy:     at ModuleWrap.<anonymous> (node:internal/modules/esm/translators:169:29)
core:test:cy:     at ModuleJob.run (node:internal/modules/esm/module_job:194:25)```

@andyhqtran I ran into this the other day and created an issue with a reproduction #28696

[mverdaguer]

Hi @lmiller1990, we've been struggling to make the @cypress/vite-plugin-cypress-esm plugin work without success for component tests in a Vue 3 + Typescript project.

An easy way to reproduce is to just scaffold the example Vue 3 app with: npm create vue@latest, and then install the plugin.

The error we are getting is this one:

We tried to add Vue to the ignoreModuleList as:

 CypressEsm({
  ignoreModuleList: ['*vue*']
})

But the problem persists.

Any ideas on how to approach it?

[lmiller1990]

Hmm can you try ignoreImportList (just a guess) https://github.com/cypress-io/cypress/tree/develop/npm/vite-plugin-cypress-esm#ignoreimportlist

I am guessing you cannot stub something defined at runtime, which could be expose.

After a lot of time/effort from many people in many projects, there is still no solution to stub/mock for ESM modules. They are sealed and immutable at the implementation level, I can't really imagine if this will ever be fully solvable.

I am not entirely sure what is the actual issue here. It looks like on line 8 of your code it does:

setup(props, {expose: __expose)`

And for some reason, the object it is trying to destructure is null. I can't think why that would be or how it's caused by this plugin.

Please try the ignoreImportList and let me know what happens. If no luck, I will think / try something out and see if I can make a recommendation.

[mverdaguer]

Hi @lmiller1990 same problem with ignoreImportList.

Keep in mind that the setup(props, {expose: __expose) is something Vue 3 automatically generates for any component, so this problem will happen to anybody using Vue 3.

[lmiller1990]

The idea of ignoreImportList was partly to let modules opt out of it. It looks like it is not getting applied:

Not sure what's next... do you have a minimal example repo so someone can clone and try debug?

[mverdaguer]

Hey @lmiller1990 I just created one for you: https://github.com/mverdaguer/cypress-vue3-esm-stubs-error/tree/main

[Crismon96]

It's great to see the effort on such an important topic. I think once there is an option to mock modules on real environments all my wishes come true. Unfortunately using the plugin on our React-Typescript-vite setup fails.

[plugin:vite:import-analysis] Failed to parse source for import analysis because the content contains invalid JS syntax. If you are using JSX, make sure to name the file with the .jsx or .tsx extension.
/Users/user/code/predium/node_modules/.vite/predium-client/deps/chunk-INT4XFS7.js:33:44
31 |    unstable_ClassNameGenerator
32 |  };
33 |  //# sourceMappingURL=chunk-INT4XFS7.js.map
   |                                             ^
34 |
    at formatError (file:///Users/user/code/predium/apps/client/node_modules/vite/dist/node/chunks/dep-68d1a114.js:44062:46)
    at TransformContext.error (file:///Users/user/code/predium/apps/client/node_modules/vite/dist/node/chunks/dep-68d1a114.js:44058:19)
    at TransformContext.transform (file:///Users/user/code/predium/apps/client/node_modules/vite/dist/node/chunks/dep-68d1a114.js:41784:22)
    at async Object.transform (file:///Users/user/code/predium/apps/client/node_modules/vite/dist/node/chunks/dep-68d1a114.js:44352:30)
    at async loadAndTransform (file:///Users/user/code/predium/apps/client/node_modules/vite/dist/node/chunks/dep-68d1a114.js:55026:29)
    at async viteTransformMiddleware (file:///Users/user/code/predium/apps/client/node_modules/vite/dist/node/chunks/dep-68d1a114.js:64430:32

and a readable error like:

> Failed to fetch dynamically imported module: http://localhost:3000/__cypress/src/@fs/Users/christophgriehl/code/predium/apps/client/src/pages/DataCollection/DataCollectionEnergyCertificateDraft.cy.tsx

I have the plugin configured with ignoreModuleList like so:
CypressEsm({ ignoreModuleList: ['*react*'], }),

The plugin seems to fail on some basic step of our project even though it is quite common and nothing unorthodox happens in our build process therefore it is difficult to attempt a repro case.

Is there anything I can do? What is the state of the plugin currently. Does someone has a working example except the PR where it got merged?

[mike-plummer]

@Crismon96 The plugin is an alpha-stage experiment so instability and errors are to be expected, but I can say for certain that it is being used successfully on several very large complex projects. The plugin relies on fairly basic methods to transform code (regexes rather than an AST) for now, so it's very possible a particular string or magic variable name in a unique third-party dependency or newer babel transform is tripping things up.

Without a reproduction case I can't provide much direct assistance, but I would recommend the following flow:

1. The error indicates that something in the plugin is generating malformed Javascript. You should be able to review the referenced file (or copy-paste into a code editor) in your browser devtools to determine what syntax error exists. If you can provide information around that it may assist with setting up a reproduction case. You should also be able to narrow down exactly what project file or third-party dependency is the problem.
2. Once you have the project file or dependency narrowed down you can use the ignoreModuleList and ignoreImportList config flags to exclude that resource from being transformed by the plugin.
3. Relaunch and see if you have success or hit another error. Repeat 😆

If you are able to narrow it down and get something working hopefully that may give you enough info to report back and open an issue with a reproduction case. Good luck!

[lmiller1990]

Just adding for future reference, someone pointed out you could experiment with using MSW to intercept a module and return mocked one. From: vitest-dev/vitest#3046 (comment)

[ZachJW34]

I was interested in this issue a while back and thought I'd play around with MSW after @lmiller1990 pointed out the viability of using MSW. I was able to get something to work (with some quirks...) if anyone is interested in checking it out cypress-vite-esm-msw-mocker and giving it a go.

The package allows you to intercept network requests and replace a modules contents with another file on disk or just a string. Works with esm modules and default exports (since the entire file gets replaced). The package is rough around the edges but could be useful!

[matthias-deschoenmacker]

Is there a solution for webpack for the same issue?

[mike-plummer]

@matthias-deschoenmacker The solution most people have used is modifying Webpack to not generate ESM modules when bundling for your Component Tests. There is a config option that allows you to customize the Webpack config applied to your project. That would cover situations where your code is bundled as ESM - if you're somehow pulling in third-party libs that are prebundled as ESM then you have a more complex problem that is a bit outside of Cypress - you'd either need to stub a higer-level wrapper that is under your control, or restructure the import to be dynamic and use something like msw to intercept and replace like suggested above. Discord would be the right place to ask for help if you run into issues trying to implement any of these.

[illegalnumbers]

Is the consensus here that vitest will not support this then?

[lmiller1990]

Outside of the plugin (with works for a good amount of cases, did you try it?) and various other suggestions that are suggested here there is no ongoing work to enable ESM mocking/stubbing in Cypress / Vite integration.

Does Vitest (eg, the Node.js test runner powered by Vite: https://vitest.dev/) even support this? My understanding is the ESM is designed specifically to not be mutable - the core issue is we are all trying to do something the spec specifically disallows (tinkering with modules).