Improve dev experience with HMR and Mock

[apiel]

HMR

A thing missing with Deno, is the possibility to implement hot module reloading. HMR is widely use in JS development but right now is not really possible to implement it properly with Deno. Would be great to have something like:

• Deno.hotModuleReload('./your_module.ts')
• or something to clear the module cache Deno.clearCacheImport('./your_module.ts')

I started to try to implement the second solution, because it sound easier to me, but since I never develop with Rust before this weekend, it is not easy :p https://github.com/apiel/deno/pull/1/files This still work in progress, I need to find a way to access the modules from ops op_clear_cache_import. (and of course need to unit test and clean up the code) By the way, I am scared that this proposal do not get accepted by the Deno team, anyhow I will still give a try.

Another option, would be to give us the possibility to provide our own module loader, but this would be much more complicated. Also I think your module loader is really great and much more efficient as it is written in Rust.

Mock

Something amazing would be to include a way to mock module in Deno. Since there is nothing like Jest in Deno, it's very hard to mock our code unless we use DI, but not everybody is using this pattern. Would be great to have:

• Deno.mock('./your_module.ts', { here: () => the_mock })
• Deno.mockRestore('./your_module.ts')

[kitsonk]

Duplicate of #4323

[apiel]

Not 100% duplicated cause there is other topic: HMR, Mock and versioning :p

HMR related to #442

[nayeemrmn]

@apiel The versioning is also a duplicate of many issues. Please remove points about TSC and versioning, and improve the title.

You can reload a module by dynamically importing it with a unique hash:

let n = 0;
import(`./your_module.ts#${n++}`)

Mocking: Could you elaborate on how this works? If I run Deno.mock('./your_module.ts', { here: () => the_mock }) at runtime even though modules have already been loaded, what would that do?

[apiel]

@nayeemrmn does the versioning issues cover as well security possible problem?

[apiel]

For the mocking example, I would suggest you to look at the jest documentation: https://jestjs.io/docs/en/jest-object#jestmockmodulename-factory-options

Mocking on run-time is necessary to be able to set different test case. The way jest is working is very complicated (or at least from what I remember), in background, it transform your code by moving the import and mocking function around. This might have some side effect of some variable not available: The module factory of jest.mock() is not allowed to reference any out-of-scope variables. So having a mocking feature build-in inside Deno, would make our life much easier.

Concerning the dynamic import, you suggestion is not always working. I was using it there https://github.com/apiel/adka/blob/dd3e62dacc89ce219a0b4590bd629768890b18d8/generatePages/generatePages.ts I realized that sometime even with a new hash, the import was still using the old cache. I had to reload several time to get it work. Also, this method, doesn't allow you to reload the child modules, see following example:

import { writeFileStr } from "https://deno.land/std/fs/write_file_str.ts";

async function test(n: number) {
  const file = "./file.ts";
  const child = "./child.ts";

  await writeFileStr(
    file,
    `import {withGetter} from './child.ts';
     console.log('first level', ${n});
     console.log(withGetter());`
  );
  await writeFileStr(
    child,
    `console.log('second level', ${n});
    export function withGetter() { return 'with getter ' + ${n}; }`
  );
  await import(`${file}?${+new Date()}${Math.random()}.ts`);
}

async function main() {
  await test(1);
  await test(2);
  await test(3);
}

main();

The output is:

Compile file:///home/alex/dev/deno/test/dynimport/test.ts
Compile file:///home/alex/dev/deno/test/dynimport/file.ts?15916191914120.5116619258703794.ts
second level 1
first level 1
with getter 1
Compile file:///home/alex/dev/deno/test/dynimport/file.ts?15916191918550.15545562533310453.ts
first level 2
with getter 1
Compile file:///home/alex/dev/deno/test/dynimport/file.ts?15916191922950.7464383962241257.ts
first level 3
with getter 1

[nayeemrmn]

Mocking on run-time is necessary to be able to set different test case. The way jest is working is very complicated (or at least from what I remember), in background, it transform your code by moving the import and mocking function around. This might have some side effect of some variable not available: The module factory of jest.mock() is not allowed to reference any out-of-scope variables. So having a mocking feature build-in inside Deno, would make our life much easier.

As you can see, this works with require() because it's used at runtime. You have to propose how this would work with ES modules.

Concerning the dynamic import, you suggestion is not always working.

Works for me.

Also, this method, doesn't allow you to reload the child modules

Could you describe your use case more? This all seems like a misuse of the module system. People often incorrectly want to use it for some kind of resource management, is that the case here?

[apiel]

I want to be able to implement an application that reload module while I am changing codes. For example, if I build a SSR server, I update some code on my site, I want to be able to reload only the changed modules and not the whole server. Reloading the module take a fraction of second (at least if we dont do type check with TSC), while reloading the server take few seconds.

Even if a HTTP server is the most common use-case, it could be as well for a MQTT broker, ZigBee middleware, ... what ever programme that get time to restart or that need to keep communication (connection) alive during development process.

Some other example in other world but just to show you that's common practice. E.g:

• in flutter, if you change a view of the application, only the changed element change, instead to reload the whole app (the same with react native)
• even with Kotlin and android sometime they manage to only reload part of the app.
• in python https://docs.python.org/3/library/importlib.html#importlib.reload
• in nodejs delete require.cache()

[apiel]

As you can see, this works with require() because it's used at runtime. You have to propose how this would work with ES modules.

If you scroll a bit down, it is jest.mock is also working with import:

The idea, would be to update the module on the fly with mocked value, but still keep the original state on the side, to be able unmock the module when we are finish with the test.

[apiel]

Related to #5548

[apiel]

Implementing such mocking and invalidating of the cache is not really possible with instantiate_module from v8. The only way to solve this, would be by transpiling the code to replace the import with a custom module loader (like require in nodejs). Maybe one day deno will let us doing this out of the box, but till then, we would have to transpile the code before to send it to Deno.