Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to invalidate ES module cache #1399

Closed
mfidemraizer opened this issue Jul 27, 2018 · 14 comments
Closed

How to invalidate ES module cache #1399

mfidemraizer opened this issue Jul 27, 2018 · 14 comments

Comments

@mfidemraizer
Copy link

@mfidemraizer mfidemraizer commented Jul 27, 2018

  • 10.x:
  • Linux Mint 18.3:
  • runtime:

Hi!

ES modules docs state:

require.cache is not used by import. It has a separate cache.

Where's that separate cache? I need to invalidate it as it can be done in CommonJS modules for testing purposes.

How can I access that cache? Or, how can I completely invalidate the module cache?

Thank you in advance!

@devsnek
Copy link
Member

@devsnek devsnek commented Jul 27, 2018

the es module cache is designed to be immutable. we are working on hooks to allow mocking and such but raw access to the cache will never be available.

@mfidemraizer
Copy link
Author

@mfidemraizer mfidemraizer commented Jul 27, 2018

@devsnek

Oops! And does changing the file:// URI break the cache? I mean, I could add a random querystring parameter, since what I want to invalidate starts from a dynamic import.

In fact, I've implemented ES module mocking already but it fails to work in some scenarios because of caching:

const httpApiInitP = importWithMocks (
         '../../../../src/init.mjs',
         import.meta.url,
         [
            `../../../../../shared/src/cache/collectionOps.mjs`,
            `./mocks/collectionOpsMock.mjs`
         ]
      )

Also, does this WIP hooks are available in nightly builds? I really need an approach to mock modules or I'll need to face a very disturbing refactor in my project codebase.

@devsnek
Copy link
Member

@devsnek devsnek commented Jul 27, 2018

@mfidemraizer es modules are still in development so i wouldn't recommend migrating your project over to them just yet. Mocking/development are high on our list of things to support so don't worry about not having that ability in the future.

I could add a random querystring parameter, since what I want to invalidate starts from a dynamic import.

Yeah that's fine. We use the full URL internally (including query and hash) for the cache key.

@mfidemraizer
Copy link
Author

@mfidemraizer mfidemraizer commented Jul 27, 2018

@devsnek Nice (about both hooks with mocking and the thing of URLs).

My concern with randomizing URLs is I'm not breaking an entire dependency tree if I'm not mistaken: say there's a module A which mocks some import: I need to randomize URLs for all modules having the mocked module dependency too. Right?

@devsnek
Copy link
Member

@devsnek devsnek commented Jul 27, 2018

@mfidemraizer yeah nothing will depend or know about the module that got re-imported. again you will just have to wait for us to figure out how to safely expose the behaviour for this kinda stuff. you can follow along at https://github.com/nodejs/modules

@devsnek
Copy link
Member

@devsnek devsnek commented Jul 28, 2018

gonna close this -- please reopen if you feel your question hasn't been answered

@devsnek devsnek closed this as completed Jul 28, 2018
@kimamula
Copy link

@kimamula kimamula commented Mar 10, 2019

We use the full URL internally (including query and hash) for the cache key.

Probably I am misunderstanding something but I cannot confirm this to be true.
I have tested the caching behavior of ESModules using Node.js v11.11.0 as follows.

// esm.mjs 
console.log('Hello, ESModules!');
// import-esm.mjs
import './esm?query=1';
import './esm?query=2';
$ node --experimental-modules import-esm
(node:40370) ExperimentalWarning: The ESM module loader is experimental.
Hello, ESModules!

As shown, Hello, ESModules! is logged only once, which indicates esm.mjs is loaded only once even if the module is imported twice with different queries.

@kimamula
Copy link

@kimamula kimamula commented May 2, 2019

Confirmed that the behavior I described above is fixed in Node.js v12.0.0.

@marxangels
Copy link

@marxangels marxangels commented Mar 7, 2021

the es module cache is designed to be immutable. we are working on hooks to allow mocking and such but raw access to the cache will never be available.

Any updates for 2021? @devsnek

@DAmNRelentless
Copy link

@DAmNRelentless DAmNRelentless commented Aug 17, 2021

I don't know if this is still relevant for anyone but here is my current solution.
I am still searching for a better way to do this.
What I am doing right now is cache-busting which causes a memory leak but it's better than rewriting everything.
I rely on the "type": "module" option.
https://ar.al/2021/02/22/cache-busting-in-node.js-dynamic-esm-imports/

@bcomnes
Copy link

@bcomnes bcomnes commented Jan 5, 2022

The issue with the cache busting query string is that it doesn't reload the entire tree, so if the thing you are trying to re-import or reload imports anything else, those nested imports will not be re-read.

@bcomnes
Copy link

@bcomnes bcomnes commented Jan 7, 2022

I found a workaround that might work for people under some circumstances. If you need to re-load an esm module, as well as its entire dependency tree, you can spawn a worker and import the module in there (https://nodejs.org/api/worker_threads.html has a good example), and it will be freshly re-imported every time you spawn a new worker. The other advantage is it avoids the memory leak that re-importing a module with a new query string in the same thread.

@frank-dspeed
Copy link

@frank-dspeed frank-dspeed commented Apr 24, 2022

@bcomnes while that is true i wonder is that really faster then restarting the node process and how much faster

@bcomnes
Copy link

@bcomnes bcomnes commented Apr 24, 2022

If you can get away with restarting the whole process, then that would work similarly. I’m not sure what’s more performant, I haven’t tested.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

7 participants