xdm webpack loader doesn’t work with webpack cli #11

remcohaszing · 2021-02-24T10:45:09Z

The issue can be reproduced from within this repository. I use Webpack 4, but the same issue can be reproduced using Webpack 5.

In this repository, add the following files:

webpack.config.cjs:

module.exports = {
  module: {
    rules: [
      {
        test: /\.mdx?$/,
        loader: require.resolve('./webpack.cjs')
      }
    ]
  }
};

src/index.js:

import './foo.md'

src/foo.md:

<Hello />

Now running the following command yields an error:

$ yarn webpack --mode production

The first time the following error is thrown:

(node:619531) UnhandledPromiseRejectionWarning: TypeError [ERR_VM_DYNAMIC_IMPORT_CALLBACK_MISSING]: A dynamic import callback was not specified.
    at exports.importModuleDynamicallyCallback (internal/process/esm_loader.js:34:9)
    at Object.module.exports (/home/remco/Projects/xdm/webpack.cjs:14:3)
    at LOADER_EXECUTION (/home/remco/Projects/xdm/node_modules/loader-runner/lib/LoaderRunner.js:132:14)
    at runSyncOrAsync (/home/remco/Projects/xdm/node_modules/loader-runner/lib/LoaderRunner.js:133:4)
    at iterateNormalLoaders (/home/remco/Projects/xdm/node_modules/loader-runner/lib/LoaderRunner.js:250:2)
    at Array.<anonymous> (/home/remco/Projects/xdm/node_modules/loader-runner/lib/LoaderRunner.js:223:4)
    at runCallbacks (/home/remco/Projects/xdm/node_modules/enhanced-resolve/lib/CachedInputFileSystem.js:27:15)
    at /home/remco/Projects/xdm/node_modules/enhanced-resolve/lib/CachedInputFileSystem.js:200:4
    at /home/remco/Projects/xdm/node_modules/graceful-fs/graceful-fs.js:123:16
    at FSReqCallback.readFileAfterClose [as oncomplete] (internal/fs/read_file_context.js:63:3)
(Use `node --trace-warnings ...` to show where the warning was created)
(node:619531) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 2)
(node:619531) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

After a retry, this error is thrown:

(node:620590) UnhandledPromiseRejectionWarning: TypeError: Invalid host defined options
    at Object.module.exports (/home/remco/Projects/xdm/webpack.cjs:14:3)
    at LOADER_EXECUTION (/home/remco/Projects/xdm/node_modules/loader-runner/lib/LoaderRunner.js:132:14)
    at runSyncOrAsync (/home/remco/Projects/xdm/node_modules/loader-runner/lib/LoaderRunner.js:133:4)
    at iterateNormalLoaders (/home/remco/Projects/xdm/node_modules/loader-runner/lib/LoaderRunner.js:250:2)
    at Array.<anonymous> (/home/remco/Projects/xdm/node_modules/loader-runner/lib/LoaderRunner.js:223:4)
    at runCallbacks (/home/remco/Projects/xdm/node_modules/enhanced-resolve/lib/CachedInputFileSystem.js:27:15)
    at /home/remco/Projects/xdm/node_modules/enhanced-resolve/lib/CachedInputFileSystem.js:200:4
    at /home/remco/Projects/xdm/node_modules/graceful-fs/graceful-fs.js:123:16
    at FSReqCallback.readFileAfterClose [as oncomplete] (internal/fs/read_file_context.js:63:3)
(Use `node --trace-warnings ...` to show where the warning was created)
(node:620590) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 2)
(node:620590) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

Afterwards the errors seem to alternate randomly.

I did not run into this issue when using koa-webpack. This may explain why this issue isn’t caught by any tests.

The text was updated successfully, but these errors were encountered:

wooorm · 2021-02-24T11:07:47Z

Are you on node 12+, as noted in the readme?

It seems to work for this user. And in the tests here. Although both don’t use the webpack CLI

wooorm · 2021-02-24T11:10:44Z

The enhanced-resolve in your stack trace looks a bit weird. Where’s that from?

remcohaszing · 2021-02-24T11:28:36Z

I’m using Node v14.16.0.

enhanced-resolve is a webpack dependency.

$ yarn why enhanced-resolve     
yarn why v1.22.5
[1/4] 🤔  Why do we have the module "enhanced-resolve"...?
[2/4] 🚚  Initialising dependency graph...
[3/4] 🔍  Finding dependency...
[4/4] 🚡  Calculating file sizes...
=> Found "enhanced-resolve@4.5.0"
info Reasons this module exists
   - "webpack" depends on it
   - Hoisted from "webpack#enhanced-resolve"
info Disk size without dependencies: "276KB"
info Disk size with unique dependencies: "468KB"
info Disk size with transitive dependencies: "1004KB"
info Number of shared dependencies: 10

Since this is a dependency of webpack, not webpack-cli, I don’t think this is the issue.

I suppose I could call webpack using A NodeJS script instead of webpack-cli, but using webpack-cli is the most basic usage IMO.

webpack/webpack-cli#1274 and webpack/webpack-cli#1622 seem related, but they apply to usage of ESM in webpack configs, not loaders.

wooorm · 2021-02-24T11:51:32Z

Can you try running this with the DISABLE_V8_COMPILE_CACHE=1 flag?

wooorm · 2021-02-24T11:53:51Z

It’s mention in that thread and it worked for me.

I’m also guessing that this might be a problem in this project, but not when xdm is in node_modules?

wooorm · 2021-02-24T11:54:59Z

This seems to be the root: zertosh/v8-compile-cache#30

remcohaszing · 2021-02-24T12:32:08Z

I initially tried this in a separate project, but I figured reproducing it inside the repository might be convenient. The issue exists there as well.

Setting DISABLE_V8_COMPILE_CACHE=1 seems to work.

wooorm · 2021-02-24T12:50:18Z

Quite perplexing to me that these tools focussed on ESM, such as webpack and snowpack, do not support ESM 🙈

remcohaszing · 2021-02-24T13:11:05Z

I actually expect the situation to get even worse in the JavaScript ecosystem as more and more packages will start using ESM, whilst others won’t even support it yet. It will be better in the end though. 😄

Related-to GH-11. Closes GH-12.

chenxsan · 2021-02-25T14:13:55Z

Or rewrite lib/webpack-loader.js in CommonJS? I'm afraid It would take a while for webpack to support ESM in loaders. DISABLE_V8_COMPILE_CACHE is not a good idea in my opinion, it would be my last resort.

remcohaszing · 2021-02-25T14:54:01Z

The loader itself is using CommonJS. The issue is that Webpack loaders can’t use any dependencies that use ESM if they are used with webpack-cli.

In other words: to fix this, would mean all of xdm would need to be converted to CommonJS. The real issue is upstream in v8-compile-cache.

wooorm · 2021-02-25T15:04:04Z

Thanks for chiming in @chenxsan! But it does seem to be somewhere in what’s webpack using (#11 (comment)) that the real issue is. Because import() should be allowed in a CJS loader imo, but that issue prevents it

chenxsan · 2021-02-26T01:07:17Z

Seems v8-compile-cache is widely used by famous tools like eslint, webpack-cli, parcel, gatsby etc., so all of them will likely to have problem like this if we want to deal with ESM. I know Node.js 10 will be end of life soon, but the world doesn't seem be to ready for ESM yet even it's out for many years.

ChristianMurphy · 2021-02-26T05:03:27Z

Seems v8-compile-cache is widely used by famous tools like eslint, webpack-cli, parcel, gatsby etc., so all of them will likely to have problem like this if we want to deal with ESM

Yes, they likely will, which is another reason why this should be addressed in v8-compile-cache.

I know Node.js 10 will be end of life soon, but the world doesn't seem be to ready for ESM yet even it's out for many years.

The transition to ESM is happening across the JS community, not just here (for example sindresorhus/meta#15)

wooorm · 2021-02-26T07:23:23Z

I know Node.js 10 will be end of life soon, but the world doesn't seem be to ready for ESM yet even it's out for many years.

I don’t think we’d be ready in two years either. I feel like the push and having to move is what’s surfacing these things, whether that’s now or another moment in time!

Still, all the issue trackers are going to be on fire soon 😅

wooorm · 2021-03-07T17:19:17Z

Closing as it’s not something we can solve here 🤷‍♂️

devmattrick · 2021-07-31T04:45:12Z

Sorry for the bump, but if you added a bundler like rollup as a build step, wouldn't it change it to a CJS compatible format? I understand this would be a major architectural change, but I think it'd work?

ChristianMurphy · 2021-07-31T05:08:03Z

See unifiedjs/unified#121, not just xdm but all layers of unified are moving to pure ESM.
Most tools have already added support for ESM or are in the process of doing so https://gist.github.com/sindresorhus/a39789f98801d908bbc7ff3ecc99d99c

We've experimented with mixed ESM and CJS in the past, it didn't go so well remarkjs/react-markdown#518
Tools are better equipped to handle pure ESM packages, than CJS and ESM fusion ones.

devmattrick · 2021-07-31T05:14:10Z

Gotcha, I appreciate the quick response. ESM is definitely the future, so I suppose tooling should adjust to support it. Thanks for your help!

remcohaszing mentioned this issue Feb 24, 2021

Document the workaround described for webpack-cli #12

Merged

wooorm pushed a commit that referenced this issue Feb 24, 2021

Add note on workaround for webpack-cli

a096d94

Related-to GH-11. Closes GH-12.

wooorm closed this as completed Mar 7, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

xdm webpack loader doesn’t work with webpack cli #11

xdm webpack loader doesn’t work with webpack cli #11

remcohaszing commented Feb 24, 2021

wooorm commented Feb 24, 2021

wooorm commented Feb 24, 2021

remcohaszing commented Feb 24, 2021

wooorm commented Feb 24, 2021

wooorm commented Feb 24, 2021

wooorm commented Feb 24, 2021

remcohaszing commented Feb 24, 2021

wooorm commented Feb 24, 2021

remcohaszing commented Feb 24, 2021

chenxsan commented Feb 25, 2021

remcohaszing commented Feb 25, 2021

wooorm commented Feb 25, 2021

chenxsan commented Feb 26, 2021

ChristianMurphy commented Feb 26, 2021

wooorm commented Feb 26, 2021 •

edited

Loading

wooorm commented Mar 7, 2021

devmattrick commented Jul 31, 2021

ChristianMurphy commented Jul 31, 2021

devmattrick commented Jul 31, 2021

xdm webpack loader doesn’t work with webpack cli #11

xdm webpack loader doesn’t work with webpack cli #11

Comments

remcohaszing commented Feb 24, 2021

wooorm commented Feb 24, 2021

wooorm commented Feb 24, 2021

remcohaszing commented Feb 24, 2021

wooorm commented Feb 24, 2021

wooorm commented Feb 24, 2021

wooorm commented Feb 24, 2021

remcohaszing commented Feb 24, 2021

wooorm commented Feb 24, 2021

remcohaszing commented Feb 24, 2021

chenxsan commented Feb 25, 2021

remcohaszing commented Feb 25, 2021

wooorm commented Feb 25, 2021

chenxsan commented Feb 26, 2021

ChristianMurphy commented Feb 26, 2021

wooorm commented Feb 26, 2021 • edited Loading

wooorm commented Mar 7, 2021

devmattrick commented Jul 31, 2021

ChristianMurphy commented Jul 31, 2021

devmattrick commented Jul 31, 2021

wooorm commented Feb 26, 2021 •

edited

Loading