Hot reloading does not reload all files

[brianlaframboise]

(Created from #823)

While hot reloading will reload a changed file, it does not appear that changes are actually used. For example, a changed file in the paths/ directory will be reloaded. However, those changes have no effect until the _.context.ts file is reloaded as well.

Steps to reproduce

Setup

Have a file structure like so:

counterfact/
    paths/
        _mocks/
            my-custom-code.ts
        _.context.ts

In _.context.ts:

import { CustomClass } from './_mocks/my-custom-code';

console.log("_.context.ts loaded");
// Do stuff with CustomClass

In ./_mocks/my-custom-code:

console.log("Custom code loaded");

const value = "Some value that's used in generating a mock response";

class CustomClass {
  // Use value in here somehow as part of generating a response
}

export { CustomClass }

Trigger

1. Run counterfact (ex: npx counterfact spec.openapi.yaml counterfact)
2. Call counterfact api, see the "Some value..." in the HTTP response
3. Modify the custom code string to be something else ("const value = "Changed!")
4. Note that Custom code loaded is logged
5. Make a request to counterfact. Note that the new value is not returned

However, if I trivially modify the _.context.ts file (touch it, add a comment, etc), I do see counterfact log the _.context.ts loaded message. If I then make a new HTTP request, I see the new value in the mock http response.

[pmcelhaney]

Thanks. In theory we should trace dependencies and reload only the affected files. But since restarting is so fast I'm going to try reloading all of the files, no matter which one changed. That should solve the problem.

When Node officially supports VM modules I'll do it the "right" way.

[pmcelhaney]

Working on this now. I'm not sure if I'll have time this weekend, but it should be fixed by April 21 at the latest.

[pmcelhaney]

Almost there. I just want to make sure the server pauses while things are loading so we don't have any race conditions.

[pmcelhaney]

It looks like we're going to have to trace dependencies and only reload the affected files. Otherwise the refresh will take too long for large codebases.

I don't think tracing dependencies will be particularly hard, it's just going to take a bit more time to implement.

[pmcelhaney]

Most of the code for tracing dependencies is done, so I should have this wrapped up in a few days.

[pmcelhaney]

Finished the code to trace dependencies. It's not working because imports are cached. In other words if we have a file that contains:

import { x } from "./x.js";

Then if we change x.js and reload the file, Node will use the original copy of x.js. And there's no way to invalidate the cache.

I can think of three ways to possibly work around it:

1. Use child processes somehow.
2. Rewrite imports to a cache-busting query string.
3. Always compile to CJS so that we can invalidate the cache.

Of these, the third seems like the simplest. Since I figured out a way use require() from ESM, and we're already invalidating the cache every time we require() an updated file, that might work with very little modification.

[pmcelhaney]

Okay, I think this will actually work today as long as your output directory has a package.json with {type: "CommonJS"}.

Which is fine as long as you're not intending the mock code to be part of a larger package.

I would rather it "just work" for everyone, so I think what I'm going to have to do is emit ".cjs" files from the transpiler. The transpiler will also have to rewrite references to local ".js" files to ".cjs".

[pmcelhaney]

Finally got it working with #879! Will put out a release shortly.

Thank you for the feedback. This is an important fix.

[pmcelhaney]

Fixed in 0.40.0