Web Workers

[LukiFr]

Hi,
is there any possibility to run web workers in Remix project?

[Mxs2019]

@nyoung697 Were you able to get shopify web worker react working with remix? Curious if there is any sample code. That interface looks super nice to work with.

[nyoung697]

I haven't used the Shopify web workers. Just a plain webworker wrapped in the webworker-promise library. We actually recently switched away from Remix and are just using Vite, which was a bit easier to bundle workers, but this is what we had done with Remix:

This is the webworker.ts file that was built with the esbuild command above and the output directory being the public/webworks folder:

import type { TypedArray } from "@kitware/vtk.js/types";
import registerWebworker from "webworker-promise/lib/register";

export interface ILoadFromCache {
  points: TypedArray;
  polys: TypedArray;
  lines: TypedArray;
  verts: TypedArray;
}

registerWebworker(async (message) => {
  const cacheUrl = message;
  console.log("Message received in worker: cacheUrl: ", cacheUrl);

  const dataBuffer = await getDataBufferFromCache(cacheUrl);

  if (!dataBuffer) {
    return null;
  }
  
  ...expesive cacluations happening here...

  const loadFromCache: ILoadFromCache = {
    points: polyData.getPoints().getData() as TypedArray,
    polys: polyData.getPolys().getData() as TypedArray,
    lines: polyData.getLines().getData() as TypedArray,
    verts: polyData.getVerts().getData() as TypedArray,
  };

  return new registerWebworker.TransferableResponse(loadFromCache, [
    loadFromCache.points.buffer,
    loadFromCache.polys.buffer,
    loadFromCache.lines.buffer,
    loadFromCache.verts.buffer,
  ]);
});

Then on the client side, we had a function like this that would get the data from the webworker:

import WorkerPromise from "webworker-promise";
import type { ILoadFromCache } from "workers/load_from_cache";

// Create worker
let worker = new WorkerPromise(
  new Worker(new URL("/workers/load_from_cache.js", import.meta.url), {
    type: "module",
  })
);

export async function loadFromCache(cacheUrl: string) {
  try {
    // Load data in worker from cache
    const cachedData = (await worker.postMessage(cacheUrl)) as ILoadFromCache;

    return cachedData;
  } catch (err) {
    console.error("loadFromCache: ", err);
    throw err;
  }
}

[JoshMatthew]

Is Partytown a good solution to this as well? https://partytown.builder.io/remix

[nyoung697]

Thanks Sergio!

What do you think of doing something like this? Add this script to package.json:

"build:workers": "esbuild ./workers/*.js --bundle --outdir=./public/workers"

Then just put all the worker files in the workers directory, and on build, it bundles and outputs them to the public folder.

I tested it out in my project, and it allows me to import other libraries into my worker (like webworker-promise).

I'm not that well versed in build tools though, so I might be overlooking something.

[sergiodxa]

That should work great if you need to import other parts of your code inside your workers.

[sergiodxa]

Yes, create a .js file inside the public folder and follow https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API/Using_web_workers to load it

Remember to only use the WebWorker client-side, so it must happen in entry.client, inside useEffect or inside an event handler.

[LukiFr]

Thank you so much! So simple..

[nyoung697]

@sergiodxa, have you found other solutions aside from putting the worker in the public folder?

If I put my worker file in the public folder, I cannot import any packages. I'm assuming this has something to do with code splitting or how remix is bundling assets.

It would be nice to have the worker file live along side my component.

I found this library: https://www.npmjs.com/package/@shopify/react-web-worker

I haven't really looked into the inner workings of it, but it seems to do what I'm looking for.

Just curious if you have found a better solution?

[sergiodxa]

The problem is that if you import the worker code you'll get back whatever the worker exports, but web workers need a path to a JS file to be initialized, to be able to import workers Remix compiler should support giving you a worker instance or at least the path and also compiling the code so you can import other code.