webpack plugin architecture does not guarantee a plugin will be able to require components from the same version of webpack

[boutell]

What is the current behavior?

If a project and one of its npm dependencies, let's call it nifty-cms, each depend on two different versions of webpack, i.e. 5 (project level) and 4 (nifty-cms), then npm and yarn will both do the reasonable thing and install webpack twice, with node_modules/nifty-cms/node_nodules containing webpack 4 and node_modules/webpack containing webpack 5. So far so good.

However, if nifty-cms also depends on a plugin — let's say it's vue-loader — and the project does not, then vue-loader will be installed as node_modules/vue-loader. Again, so far so good. Hoisting is normal.

But here's where we get in trouble: unfortunately, when webpack 4 in nifty-cms loads vue-loader, vue-loader then tries to require various files back from webpack itself, for instance:

const BasicEffectRulePlugin = require('webpack/lib/rules/BasicEffectRulePlugin')

At this point, npm (or yarn) sees that vue-loader was loaded from node_modules/vue-loader and loads node_modules/webpack.

But, this is not the version of webpack that instantiated the loader. So the user gets the following error indicating the discrepancy:

The 'compilation' argument must be an instance of Compilation

After puzzling over this for a while I can see two possible resolutions:

1. All webpack plugin releases could have a peerDependency specifying only one major webpack version, even if they are actually compatible with more than one with no code changes. (vue-loader is compatible with webpack 3, 4, and 5 all at the same time.) This might help, depending on whether npm/yarn actually pay attention to peer dependencies with regard to hosting. And it doesn't require an API change in webpack. But, it requires a lot of redundant publication of modules. And the need for it would be easy to miss.

2. webpack could provide a webpackRequire function on the compiler object passed to plugins which invokes require from webpack. I think this is the better solution. Unfortunately for this solution to really solve the problem it would need to be universally used. It would be difficult to stomp out the practice of requiring webpack/something when it works any time a project has only one version of webpack in it.

If the current behavior is a bug, please provide the steps to reproduce.

Please see the scenario above.

What is the expected behavior?

Projects and their dependent modules should be able to depend on and use different major versions of webpack independently without encountering the following error or its underlying cause:

The 'compilation' argument must be an instance of Compilation

Specifically, webpack plugins should be able to reliably require additional resources from the same webpack version that instantiated them.

Workaround:

A module in the position of nifty-cms can set webpack as a peer dependency instead, and support more than one version of webpack. However work must still immediately be done every time a new version of webpack comes out, and this could become untenable if other webpack plugin dependencies of nifty-cms only work with a single version of webpack in a given release. A further workaround, then, is for nifty-cms to publish nifty-cms-webpack-4-something-loader, nifty-cms-webpack-5-something-loader, etc. which each just require and export a specific version of something-loader. That's a lot of npm packages.

Other relevant information:
webpack version: 4 and 5
Node.js version: any
Operating System: N/A

This issue has been seen in the wild in various projects.

[sokra]

Yes I'm aware of this problem. Since webpack 5 the Compiler object has a webpack property, which contains the webpack exports of the current version running. So plugins no longer need to require("webpack") or have a webpack dependency at all (a peerDependency might be still a good idea to communicate which webpack version is supported).

Webpack 4 doesn't support it, but you can do const webpack = compiler.webpack || require("webpack") for that case.

class MyPlugin {
  apply(compiler) {
    const webpack = compiler.webpack || require("webpack");
    compiler.hooks.compilation.tap("MyPlugin", compilation => {
      // The following like will fail with a wrong webpack version.
      webpack.javascript.JavascriptModulesPlugin.getCompilationHooks(compilation);
    });
  }
}

But accessing private modules like require('webpack/lib/rules/BasicEffectRulePlugin') is not possible this way, but accessing private modules isn't supported anyway.

[boutell]

Hmm, so this almost sounds like a perfect resolution, until that last line (: Is vue-loader doing something wrong — should it gain access to BasicEffectRulePlugin and its numerous friends some other way?

…

On Fri, Feb 5, 2021 at 3:55 PM Tobias Koppers ***@***.***> wrote: Yes I'm aware of this problem. Since webpack 5 the Compiler object has a webpack property, which contains the webpack exports of the current version running. So plugins no longer need to require("webpack") or have a webpack dependency at all (a peerDependency might be still a good idea to communicate which webpack version is supported). Webpack 4 doesn't support it, but you can do const webpack = compiler.webpack || require("webpack") for that case. class MyPlugin { apply(compiler) { const webpack = compiler.webpack || require("webpack"); compiler.hooks.compilation.tap("MyPlugin", compilation => { // The following like will fail with a wrong webpack version. webpack.javascript.JavascriptModulesPlugin.getCompilationHooks(compilation); }); }} But accessing private modules like require('webpack/lib/rules/BasicEffectRulePlugin') is not possible this way, but accessing private modules isn't supported anyway. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#12606 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAH27NLBA4ZE3UFANTWD6TS5RLKRANCNFSM4XFMOS2Q> .

-- THOMAS BOUTELL | CHIEF TECHNOLOGY OFFICER APOSTROPHECMS | apostrophecms.com | he/him/his

[sokra]

Actually it should not use that at all. Afaik it tries to copy the rules for CSS, JS, etc for their blocks, but webpack has a better mechnism to run a file as if it is a .css, .js, .ts file: !=!

Example: for a my-file.vue file with

<style>
...
</style>

<script>
...
</script>

the vue-loader can generate code like that:

import "./my-file.vue.css!=!vue-loader/get-block?range=1-3!./my-file.vue";
import "./my-file.vue.js!=!vue-loader/get-block?range=4-6!./my-file.vue";

Where vue-loader/get-block is a loader that extract the content of a block from a range.

The inline syntax ./my-file.vue.css!=! will treat the result of vue-loader/get-block?range=1-3!./my-file.vue as if it was written in ./my-file.vue.css and apply the .css rules to it.

[boutell]

Interesting. So I guess the next question is how common it is for loaders and plugins in heavy use to follow these antipatterns.

…

On Fri, Feb 5, 2021 at 4:46 PM Tobias Koppers ***@***.***> wrote: Actually it should not use that at all. Afaik it tries to copy the rules for CSS, JS, etc for their blocks, but webpack has a better mechnism to run a file as if it is a .css, .js, .ts file: !=! Example: for a my-file.vue file with <style>...</style><script>...</script> the vue-loader can generate code like that: import "./my-file.vue.css!=!vue-loader/get-block?range=1-3!./my-file.vue";import "./my-file.vue.js!=!vue-loader/get-block?range=4-6!./my-file.vue"; Where vue-loader/get-block is a loader that extract the content of a block from a range. The inline syntax ./my-file.vue.css!=! will treat the result of vue-loader/get-block?range=1-3!./my-file.vue as if it was written in ./my-file.vue.css and apply the .css rules to it. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#12606 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAH27NTND4BD2ESCUH7IZDS5RRLBANCNFSM4XFMOS2Q> .

-- THOMAS BOUTELL | CHIEF TECHNOLOGY OFFICER APOSTROPHECMS | apostrophecms.com | he/him/his

[alexander-akait]

So I guess the next question is how common it is for loaders and plugins in heavy use to follow these antipatterns.

When you need it 😄 Why do you think it is antipatterns?

[boutell]

When I used the term "antipatterns," I was referring to the importing of private modules, which Tobias indicated is not appropriate. If that is common in loaders, then it could be a long road to realistically make this work in most applications, but if it's just vue-loader that's a different story.

…

On Sat, Feb 6, 2021 at 6:44 AM Alexander Akait ***@***.***> wrote: So I guess the next question is how common it is for loaders and plugins in heavy use to follow these antipatterns. When you need it 😄 Why do you think it is antipatterns? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#12606 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAH27PYXNSTF6HBINGA7TLS5UTS3ANCNFSM4XFMOS2Q> .

-- THOMAS BOUTELL | CHIEF TECHNOLOGY OFFICER APOSTROPHECMS | apostrophecms.com | he/him/his

[boutell]

I surveyed the following loaders and plugins with a pretty basic grep search: case-sensitive-paths-webpack-plugin corejs-upgrade-webpack-plugin fork-ts-checker-webpack-plugin html-webpack-plugin mini-css-extract-plugin pnp-webpack-plugin stylelint-webpack-plugin terser-webpack-plugin babel-loader css-loader eslint-loader file-loader postcss-loader raw-loader sass-loader sass-resources-loader style-loader url-loader vue-loader vue-style-loader Of these, the only offenders that required private modules from webpack were vue-loader and html-webpack-plugin. html-webpack-plugin seems pretty official. I don't know webpack internals enough to know if it needs to be doing what it's doing.

…

On Sun, Feb 7, 2021 at 4:22 PM Tom Boutell ***@***.***> wrote: When I used the term "antipatterns," I was referring to the importing of private modules, which Tobias indicated is not appropriate. If that is common in loaders, then it could be a long road to realistically make this work in most applications, but if it's just vue-loader that's a different story. On Sat, Feb 6, 2021 at 6:44 AM Alexander Akait ***@***.***> wrote: > So I guess the next question is how common it is for loaders and plugins > in heavy use to follow these antipatterns. > > When you need it 😄 Why do you think it is antipatterns? > > — > You are receiving this because you authored the thread. > Reply to this email directly, view it on GitHub > <#12606 (comment)>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AAAH27PYXNSTF6HBINGA7TLS5UTS3ANCNFSM4XFMOS2Q> > . > -- THOMAS BOUTELL | CHIEF TECHNOLOGY OFFICER APOSTROPHECMS | apostrophecms.com | he/him/his

-- THOMAS BOUTELL | CHIEF TECHNOLOGY OFFICER APOSTROPHECMS | apostrophecms.com | he/him/his

[alexander-akait]

@boutell You need open issues in them repos, we can't fix it here, our recommend solution - using compiler.webpack (compiler.webpack can be used in loaders and plugins without problems), feel free to feedback.

New loader API decried above.

[boutell]

OK thanks. I'll take it to vue-loader. I think html-webpack-plugin does not really have this problem because their peer dependency only matches one major release of webpack.

…

On Mon, Feb 8, 2021 at 5:02 AM Alexander Akait ***@***.***> wrote: Closed #12606 <#12606>. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#12606 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAH27LHGYJE724BIANNLRTS56ZDBANCNFSM4XFMOS2Q> .

-- THOMAS BOUTELL | CHIEF TECHNOLOGY OFFICER APOSTROPHECMS | apostrophecms.com | he/him/his