-
Notifications
You must be signed in to change notification settings - Fork 26.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Webpack compat error for wasm files imported in web workers #22581
Comments
I have found a workaround by leveraging webpack asset modules to instruct webpack and next to only return the path of the digested wasm. The downside is that now one has to write the code to fetch, compile, and instantiate which is not too hard. This is how I achieved it: next.config.js module.exports = {
future: {
webpack5: true
},
webpack: (config) => {
const experiments = config.experiments || {};
config.experiments = {...experiments, asyncWebAssembly: true};
config.output.assetModuleFilename = `static/[hash][ext]`;
config.output.publicPath = `/_next/`;
config.module.rules.push({
test: /\.wasm/,
type: 'asset/resource',
})
return config
},
} worker.js import wasmPath from "../add.wasm";
onmessage = async function (e) {
const fetchPromise = fetch(wasmPath);
const { instance } = await WebAssembly.instantiateStreaming(fetchPromise);
console.log(instance.exports.add_one(10));
}; Look ma, no 3rd party dependencies. I still believe there is a bug between how next.js and webpack communicate wam modules referenced by web workers, but at least the workaround is tolerable. |
The use of webpack asset modules is a good enough workaround that I'm closing this issue |
This workaround doesn't work for me, unfortunately. I'm using I've tracked the error down to this line: https://github.com/vercel/next.js/blob/canary/packages/next/build/webpack/plugins/nextjs-ssr-import.ts#L14
EDIT: Just skipping the body of |
This skips 'require' normalization for chunks without a name, which was throwing errors when used with WASM imports. See vercel#22581.
To patch yourself without using the above PR, you can do this: next.config.js
This is importing Next internals & mucking with them so it's likely to break in the future. Hopefully we can get that patch upstreamed :) |
We use the Worker interface provided by Webpack 5 and ran into this issue when attempting to update to NextJS 11. @tomlagier 's patch appears to have fixed it, but it would be great to not have to have that in our config file. |
Thanks for the fix @tomlagier! Just for others - I had to add a |
I have also received this error although not related to wasm The error appears when I import a file from a worker that isn't a relative import. eg.
new Worker(new URL("../worker.js", import.meta.url))
// Non relative import
import * as Curry from "rescript/lib/es6/curry.js"; The config workaround above fixes the problem, as does the patch in #26372 |
@tom-sherman I am also receiving this error due to the web worker issue you describe (after upgrading to next 11). The fix by @tomlagier and patch #26372 work for me as well. |
In addition to @tomlagier's webpack: (config, { buildId, dev, isServer, defaultLoaders, webpack }) => {
// ...
config.module.rules.find(k => k.oneOf !== undefined).oneOf.unshift(
{
test: /\.wasm$/,
type: "asset/resource",
generator: {
filename: 'static/chunks/[path][name].[hash][ext]'
},
}
);
// ... Without that bit, I was getting HTTP 404 errors during execution, because for some reason my wasm file was not properly served. Now, everything seems to be working fine for me. I'm using an Emscripten-generated project using multithreading (i.e. web workers), compiled with (Just noticed that this part of the solution was already mentioned by @nickbabcock. Took me a while to figure out that I have to combine both solutions) |
In addition to the error mentioned in this issue, I also got the error mentioned in #22813 (chunk names break in a production build, leading to HTTP 404 errors). Below is my full const SSRPlugin = require("next/dist/build/webpack/plugins/nextjs-ssr-import").default;
const { dirname, relative, resolve, join } = require("path");
// Unfortunately there isn't an easy way to override the replacement function body, so we
// have to just replace the whole plugin `apply` body.
function patchSsrPlugin(plugin) {
plugin.apply = function apply(compiler) {
compiler.hooks.compilation.tap("NextJsSSRImport", compilation => {
compilation.mainTemplate.hooks.requireEnsure.tap(
"NextJsSSRImport",
(code, chunk) => {
// This is the block that fixes https://github.com/vercel/next.js/issues/22581
if (!chunk.name) {
return;
}
// Update to load chunks from our custom chunks directory
const outputPath = resolve("/");
const pagePath = join("/", dirname(chunk.name));
const relativePathToBaseDir = relative(pagePath, outputPath);
// Make sure even in windows, the path looks like in unix
// Node.js require system will convert it accordingly
const relativePathToBaseDirNormalized = relativePathToBaseDir.replace(
/\\/g,
"/"
);
return code
.replace(
'require("./"',
`require("${relativePathToBaseDirNormalized}/"`
)
.replace(
"readFile(join(__dirname",
`readFile(join(__dirname, "${relativePathToBaseDirNormalized}"`
);
}
);
});
};
}
module.exports = {
reactStrictMode: true,
headers: async () => [
"/",
"/_next/static/chunks/:slug",
].map(source => ({
source,
headers: [
{
key: 'Cross-Origin-Opener-Policy',
value: 'same-origin',
},
{
key: 'Cross-Origin-Embedder-Policy',
value: 'require-corp',
},
],
})),
webpack: (config, { buildId, dev, isServer, defaultLoaders, webpack }) => {
const ssrPlugin = config.plugins.find(plugin => plugin instanceof SSRPlugin);
if (ssrPlugin) patchSsrPlugin(ssrPlugin);
config.module.rules.find(k => k.oneOf !== undefined).oneOf.unshift(
{
test: /\.wasm$/,
type: "asset/resource",
generator: {
filename: 'static/chunks/[path][name].[hash][ext]'
},
}
);
config.output.chunkFilename = isServer
? `${dev ? "[name]" : "[name].[fullhash]"}.js`
: `static/chunks/${dev ? "[name]" : "[name].[fullhash]"}.js`;
return config;
},
} That was hard... But I'm happy that it finally works now. |
+1 to @tomlagier 's patch (after adding a |
This skips 'require' normalization for chunks without a name, which was throwing errors when used with WASM imports. See vercel#22581.
This issue has been automatically locked due to no recent activity. If you are running into a similar issue, please create a new issue with the steps to reproduce. Thank you. |
What version of Next.js are you using?
10.0.7
What version of Node.js are you using?
v14.3.0
What browser are you using?
N/A
What operating system are you using?
Ubuntu 20.04
How are you deploying your application?
N/A
Describe the Bug
I took the with-webassembly example and moved the loading and execution of wasm into a web worker (to simulate offloading of expensive functions to wasm in another thread). Unfortunately I receive the following error for both
next dev
andnext build
Which I have been unable to debug.
The project is straightforward (no additional dependencies, just plain next.js) but I'll copy the relevant bits here for posterity:
worker.js
index.js
One very important bit is that this is a webpack 5 project as webpack 5 supports web workers and wasm natively (albeit experimentally) and I've been unable to get webpack 4 working with the appropriate plugins due to #21679 .
Wasm files outside of web workers work, and web workers without wasm work, the combination of them seems to give rise to the error. It doesn't matter where the wasm module is imported in the web worker or even if it is dynamically imported or not.
Expected Behavior
For the project to be built
To Reproduce
I've created a bug repro: https://github.com/nickbabcock/next.js-wasm-worker
clone, install, and build to repro the error.
The text was updated successfully, but these errors were encountered: