Bundling with rollup

[wheelere]

I'm trying to use rollup to bundle my project, but I get Error: 'spawn' is not exported by ../../node_modules/threads/dist/index.js

Is it possible to use this package with Rollup?

[andywer]

Hey @wheelere!

The issue is probably based on the fact that no ES modules are published, but only their transpiled CommonJS equivalents.

You can first try to use rollup-plugin-commonjs and hopefully get rollup to consume the library as is.

I will also try to publish a new version that comes with both, CommonJS and ES modules. That should solve the issue as well.

[andywer]

Done. You can try to npm install threads@esm.

Let me know how it went 😉

[wheelere]

Thanks! It works!

[wheelere]

I'm still getting a build issue:
My target is browser, and the error I get there is require is undefined

src/index.ts → dist/index.js...
(!) `this` has been rewritten to `undefined`
https://rollupjs.org/guide/en#error-this-is-undefined
../../node_modules/threads/dist-es/worker/index.js
1: var __awaiter = (this && this.__awaiter) || function (thisArg, _arguments, P, generator) {
                    ^
2:     return new (P || (P = Promise))(function (resolve, reject) {
3:         function fulfilled(value) { try { step(generator.next(value)); } catch (e) { reject(e); } }
...and 1 other occurrence
../../node_modules/threads/dist-es/master/spawn.js
1: var __awaiter = (this && this.__awaiter) || function (thisArg, _arguments, P, generator) {
                    ^
2:     return new (P || (P = Promise))(function (resolve, reject) {
3:         function fulfilled(value) { try { step(generator.next(value)); } catch (e) { reject(e); } }
...and 1 other occurrence
../../node_modules/threads/dist-es/master/pool.js
1: var __awaiter = (this && this.__awaiter) || function (thisArg, _arguments, P, generator) {
                    ^
2:     return new (P || (P = Promise))(function (resolve, reject) {
3:         function fulfilled(value) { try { step(generator.next(value)); } catch (e) { reject(e); } }
...and 1 other occurrence

[andywer]

Can you add the error output for require is undefined? The output above only shows this has been rewritten to undefined, which is not ideal either, but not critical, I guess.

[andywer]

PS: Did you install threads@esm or just the latest threads? Also: Did you use rollup-plugin-commonjs and rollup-plugin-node-resolve?

I quickly tried it myself, using microbundle, and it seemed to work. Well, there was another issue (see #125), but that seems unrelated to what you are describing.

[wheelere]

The version of threads I'm using is threads@1.0.0-beta-esm.
The plugins I'm using for rollup are:
• rollup-plugin-commonjs
• rollup-plugin-node-builtins (I was seeing a shims error)
• rollup-plugin-node-resolve and
• rollup-plugin-typescript

Beyond that, my rollup.config.js file isn't very notable. Could my output format cause the issue? I'm configuring for UMD.

The require is undefined error isn't very verbose in the browser, and the line number isn't very helpful, but it's pointing to the statement:

var Implementation = typeof process !== 'undefined' && process.arch !== 'browser' && 'pid' in process
    ? require('./implementation.node').default
    : require('./implementation.browser').default;

There are a few other places in the bundled file where require is used, as well.

[wheelere]

I'm still not sure what's causing the issue with this -> undefined, but I believe the issue with require has to do with the use of require in the code when import/export is also being used. The instances of this are marked with tslint:disable no-var-requires in your code.

[andywer]

I think I might have a fix. Will get back to this tonight or tomorrow morning.

[andywer]

Try threads@1.0.0-beta-esm.1. I think this one should do *crossing fingers*

[wheelere]

That fixed the this -> undefined warnings, but the shim you're using for process is causing an error:

// shim for using process in browser
if (typeof global.setTimeout === 'function') ;
if (typeof global.clearTimeout === 'function') ;

causes

Uncaught ReferenceError: global is not defined

Rollup is still complaining, but only about the use of eval

Also, I'm still seeing an instance of require in the code:

function initTinyWorker() {
    const TinyWorker = require("tiny-worker");

[andywer]

but the shim you're using for process is causing an error:

What shim? 😅
I think I didn't add any shim.

Rollup is still complaining, but only about the use of eval

This should only be a warning. We can have a look if there is any other way around it, but that hack is required to make threads work with webpack.

[wheelere]

Oops, that's in one of my other packages!

[andywer]

Give it a try and tell me if it works 😉

Should work, though. In the feature branch with the rollup fixes I now even added an integration test that bundles a tiny sample app that uses threads via rollup and runs it in a headless Chrome.

[wheelere]

I'm still having issues 🙁
Even if I gut my project and just run the code you have, I still hit a ton of issues. First being that trying to import from threads/worker doesn't work with rollup:

[!] Error: 'expose' is not exported by ../../node_modules/threads/worker.js
https://rollupjs.org/guide/en#error-name-is-not-exported-by-module-
src/workers/worker.ts (1:9)
1: import { expose } from "threads/worker"
            ^
2:
3: expose(async function add(a, b) {
Error: 'expose' is not exported by ../../node_modules/threads/worker.js
    at error (/Users/earnest/web-components/packages/data-engine/node_modules/rollup/dist/rollup.js:9365:30)
    at Module.error (/Users/earnest/web-components/packages/data-engine/node_modules/rollup/dist/rollup.js:13272:9)
    at handleMissingExport (/Users/earnest/web-components/packages/data-engine/node_modules/rollup/dist/rollup.js:13194:21)
    at Module.traceVariable (/Users/earnest/web-components/packages/data-engine/node_modules/rollup/dist/rollup.js:13586:17)
    at ModuleScope.findVariable (/Users/earnest/web-components/packages/data-engine/node_modules/rollup/dist/rollup.js:12293:39)
    at Identifier$1.bind (/Users/earnest/web-components/packages/data-engine/node_modules/rollup/dist/rollup.js:8313:40)
    at CallExpression$1.bind (/Users/earnest/web-components/packages/data-engine/node_modules/rollup/dist/rollup.js:7949:23)
    at CallExpression$1.bind (/Users/earnest/web-components/packages/data-engine/node_modules/rollup/dist/rollup.js:10300:15)
    at ExpressionStatement$1.bind (/Users/earnest/web-components/packages/data-engine/node_modules/rollup/dist/rollup.js:7949:23)
    at Program$1.bind (/Users/earnest/web-components/packages/data-engine/node_modules/rollup/dist/rollup.js:7945:31)

even after that (if I don't bundle the worker) I hit the same issues I was hitting with Uncaught ReferenceError: global is not defined when I load my bundle in Chrome. From the looks of the bundle, it seems like global and process are not being handled from the debug dependency you have.

I'm new to all of this package management, so I'm totally at a loss.

[andywer]

If you are new to all this, let me give you a tip: Don't use rollup! 😅🙈

If you are looking for something that just works, rollup is probably not the place to look for. Have used it myself a few times and it always costed me a lot of time to get it working. It's also full of reports of badly handled edge cases that will easily break any setup and are hard to debug...

[andywer]

Use webpack or parcel or browserify or whatever. Chances are that every one of them will get you there in shorter time and with less hair pulling, I think.

[andywer]

PS: If you still wanna use rollup: There was an issue with the expose export, indeed, as in the error you shared before.

This has now been fixed: #127. Shall I publish one more updated hopefully-rollup-friendly pre-release version?

[wheelere]

That would be great! My mentor insists I use rollup for my project. I took a break from trying to figure out how to use rollup, typescript, and web workers simultaneously and worked on other areas of the project, but I'm getting back to it now, and still struggling.

Sorry for the delayed follow up!

[andywer]

Just published threads@1.0.0-beta-esm.2 (you can npm i threads@esm). Tell me if it works :)

[wheelere]

Still having issues many :(
I talked to my mentor and I will try using webpack.

[andywer]

Just out of curiosity: Is it only threads.js you are having trouble with when trying to bundle with rollup or is it other packages, too?

[wheelere]

Threads.js is the only one throwing errors at me. Most packages are working fine, but the other plugins I've tried to use to bundle a worker with rollup fail silently (that is, they compile, but the worker doesn't respond).

[andywer]

Closed by #124 🚀