New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot import wasm in web workers #7647

Open
cars10 opened this Issue Jul 3, 2018 · 10 comments

Comments

Projects
None yet
6 participants
@cars10
Copy link

cars10 commented Jul 3, 2018

Bug report

What is the current behavior?

Importing wasm in web workers throws an error.

If the current behavior is a bug, please provide the steps to reproduce.

Webpack configuration:

This is basically a merge of the worker + wasm examples.

const webpack = require('webpack')
const path = require('path')
const HtmlWebpackPlugin = require('html-webpack-plugin');
const prod = process.env.NODE_ENV === 'production'

module.exports = {
    entry: './src/js/index.js',
    output: {
        path: path.resolve(__dirname, 'dist'),
        filename: prod ? 'js/main.[chunkhash].js' : 'js/main.js',
        webassemblyModuleFilename: '[modulehash].wasm',
        globalObject: 'this'
    },
    module: {
        rules: [
            {
                test: /\.wasm$/,
                type: 'webassembly/experimental'
            }
        ]
    },
    plugins: [
        new HtmlWebpackPlugin({ template: './src/index.html' }),
        new webpack.LoaderOptionsPlugin()
    ]
};

index.js

var Worker = require("worker-loader!./worker")
var worker = new Worker()
worker.onmessage = event => {
    console.log('mainthread got:', event.data)
}

worker.js

onmessage = event => {
    import('../wasm/hello_world.wasm').then(module => {
        postMessage(module.add_one(event.data))
    })
}

This leads to :

Uncaught (in promise) TypeError: Cannot read property './src/wasm/hello_world.wasm' of undefined

But the same wasm import works when put in index.js.

What is the expected behavior?

We should be able to import wasm in workers just as we can in normal js files.

Other relevant information:
webpack version: 4.14
Node.js version: 10.5.0
Operating System: Arch
Additional tools:

@xtuc

This comment has been minimized.

Copy link
Member

xtuc commented Jul 3, 2018

Could you please provide the full stack trace?

@cars10

This comment has been minimized.

Copy link

cars10 commented Jul 3, 2018

Sure.

Uncaught (in promise) TypeError: Cannot read property './src/wasm/hello_world.wasm' of undefined
    at eval (hello_world.wasm:3)
    at Object../src/wasm/hello_world.wasm (0.e8ba8a05634a4147139d.worker.js:10)
    at __webpack_require__ (e8ba8a05634a4147139d.worker.js:732)
    at fn (e8ba8a05634a4147139d.worker.js:103)
(anonymous) @ hello_world.wasm:3
./src/wasm/hello_world.wasm @ 0.e8ba8a05634a4147139d.worker.js:10
__webpack_require__ @ e8ba8a05634a4147139d.worker.js:732
fn @ e8ba8a05634a4147139d.worker.js:103
Promise.then (async)
onmessage @ worker.js:2

Clicking on the first error in chrome ("at eval (hello_world.wasm:3)") opens "hello_world.wasm" and shows:

"use strict";
// Instantiate WebAssembly module
var wasmExports = __webpack_require__.w[module.i];
__webpack_require__.r(exports);
// export exports from WebAssembly module
for(var name in wasmExports) if(name != "__webpack_init__") exports[name] = wasmExports[name];
// exec imports from WebAssembly module (for esm order)

// exec wasm module
wasmExports["__webpack_init__"]()

With the error beeing w[module.i].

@sokra

This comment has been minimized.

Copy link
Member

sokra commented Jul 3, 2018

This isn't implemented yet...

@cars10

This comment has been minimized.

Copy link

cars10 commented Jul 5, 2018

Okay, thanks for the info.

For anyone trying to solve the same thing, i got it working like this: (see https://github.com/cars10/rust-wasm-webworker-example for an example)

webpack config

const path = require('path')
const HtmlWebpackPlugin = require('html-webpack-plugin');
const prod = process.env.NODE_ENV === 'production'

module.exports = {
    entry: './src/js/index.js',
    output: {
        path: path.resolve(__dirname, 'dist'),
        filename: 'js/main.[hash].js',
        globalObject: 'this'
    },
    module: {
        rules: [
            {
                test: /worker\.js$/,
                use: {
                    loader: 'worker-loader',
                    options: {
                        name: 'js/worker.[hash].js'
                    }
                }
            },
            {
                test: /\.wasm$/,
                type: 'javascript/auto',
                use: [
                    {
                        loader: 'file-loader',
                        options: {
                            name: 'wasm/[name].[hash].[ext]',
                            publicPath: '../'
                        }
                    }
                ]
            }
        ]
    },
    plugins: [
        new HtmlWebpackPlugin({ template: './src/index.html' })
    ],
    mode: prod ? 'production' : 'development'
};

index.js

import Worker from './worker.js';

var worker = new Worker
worker.postMessage(1)
worker.onmessage = event => {
    console.log('mainthread got:', event.data)
}

worker.js

import wasm from '../wasm/hello_world.wasm';

onmessage = event => {
    WebAssembly.instantiateStreaming(fetch(wasm))
        .then(results => console.log(results.instance.exports.add_one(12)));
    console.log('got some vent')
}
@nfrasser

This comment has been minimized.

Copy link

nfrasser commented Sep 23, 2018

I was able to use Wasm in a web worker by avoiding use of the worker-loader plugin and choosing the webworker target:

webpack.config.js

const path = require('path')

const browserConfig = {
  entry: './app.js',
  output: {
    path: path.resolve(__dirname, "dist"),
    filename: "app.js",
  },
  mode: "development"
}

const workerConfig = {
  entry: "./worker.js",
  target: 'webworker',
  output: {
    path: path.resolve(__dirname, "dist"),
    filename: "worker.js"
  },
  mode: "development",
}

module.exports = [browserConfig, workerConfig]

worker.js

// A dependency graph that contains any wasm must all be imported
// Can also import() any module that has 
// `import * as wasm from '../pkg/hello_world_bg.wasm'`
import("../pkg/hello_world_bg.wasm")
  .then(wasm => {
    self.postMessage(wasm.hello_world())
  })
  .catch(err => console.error("Error importing `hello_world_bg.wasm`:", err));

app.js

var worker = new Worker('./worker.js')
worker.addEventListener('message', function (evt) {
  console.log(evt.data) // 'Hello World'
})
@ernieturner

This comment has been minimized.

Copy link

ernieturner commented Oct 9, 2018

Is there any roadmap (or other tracking ticket) of when this might be supported natively within webpack? Or is this the ticket that should be tracked for this support?

@xtuc

This comment has been minimized.

Copy link
Member

xtuc commented Oct 9, 2018

@ernieturner see #7647 (comment), it's native in Webpack.

@royaltm

This comment has been minimized.

Copy link

royaltm commented Nov 22, 2018

@xtuc this is far from the optimal solution as we can't mangle (with [hash] or similar) the file name of the produced worker.js file, which impacts caching evasion.

@ernieturner

This comment has been minimized.

Copy link

ernieturner commented Nov 26, 2018

FWIW, when I ended up implementing this I used the webpack DefinePlugin to make the path to my WebWorker file dynamic based on the environment. In development I just set it up like this to just point to the normal webpack output path

new webpack.DefinePlugin({
  _WORKER_PATH_LOCATION_: JSON.stringify("./webpack/dist/worker.js"),
}),

And then in my production webpack config I set it up like this. My outputPublicPath has a unique version in it which gives me the ability to cache bust, but it shouldn't be too hard to manually put in a hash as part of this configuration as well

new webpack.DefinePlugin({
  _WORKER_PATH_LOCATION_: JSON.stringify(`${argv.outputPublicPath}worker.min.js`),
}),

Then in the actual code it looks like this

const worker = new Worker(_WORKER_PATH_LOCATION_);
@royaltm

This comment has been minimized.

Copy link

royaltm commented Nov 27, 2018

@ernieturner in this case I still couldn't use the "[hash]" or "[contenthash]" in worker output file name, as their values are being emitted after the build. AFAIK the DefinePlugin substitution must be provided before building. It would work however only with hashes provided up front, which I was looking to avoid.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment