[Feature] Webpack 5 cache support

[DylanPiercey]

• I'd be willing to implement this feature

Describe the user story

I am currently using this module as part of @marko/webpack. The loader for this module adds virtual modules while compiling the Marko templates.

Problem is that with the new filesystem cache in webpack 5 since those compilations are cached this module does not properly represent the virtual file system anymore.

Describe the solution you'd like

I think it'd be nice if this module would store any virtual files in the compilation in the webpack cache compilation.cache.store and retrieve them when the compilation initializes.

Describe the drawbacks of your solution

The cache api's are async and so it may slightly complicate the code.

[larixer]

I am open to this enhancement, but I lack the bandwidth to implement it myself at the moment.

[natew]

@DylanPiercey it's hard to find any docs at all for caching, I'm attempting to get some of it working now but could really use any decent examples, know of any?

[DylanPiercey]

@natew I was playing around with the API's and just inspecting by crawling through webpacks source and debugging, but for the life of me, I couldn't get it working to actually store anything in webpacks cache. At this point, I'm somewhat just waiting for some other modules to take the lead on that so I can reference.

[larixer]

@DylanPiercey What I have in mind is just to monkeypatch input and watch file systems used by webpack and skip dealing with its internals altogether, in the similar way we do in Yarn 2:
https://github.com/yarnpkg/berry/tree/master/packages/yarnpkg-fslib
It seems more futureproof solution than dealing with Webpack internals on each major Webpack release, the idea needs some validation though.

[natew]

I tried this, and tried so many different things (using memfs, unionfs and fs-monkey), and in the end couldn't get it all to play nice for various reasons, even after patching/working around a variety of things.

[jquense]

The webpack caching mechanism is a black hole...I took a few hours to explore this and it's basically impossible to discern what the code is even doing. AFAICT there is some fileSystemInfo snapshot check that results in an attempt to actually resolve files, i guess maybe before the they are pulled from the cache? I can't tell, it does seem like the virtual files are saved in the cache already but there is likely some timestamp check for freshness that fails perhaps?

[xwchris]

I think I have find the problem, I will test the code and create a pr

[VincentGarreau]

Any news regarding this issue? :)

[larixer]

Closing the issue as stale, please retest on latest webpack-virtual-modules and reopen with reproduction steps if the issue still exists.

[Zn4rK]

This issue is still present. The PR that @xwchris opened solves it for me. I probably can, in a few days, add tests and backwards compatibility for webpack 4. Would it be possible to get it merged if I do that?

[larixer]

@Zn4rK

I probably can, in a few days, add tests and backwards compatibility for webpack 4. Would it be possible to get it merged if I do that?

Yes, it will be merged if all requirements are met: quality code, working code and tests. The reason for closure of @xwchris PR:

Closing, the PR has no tests