new Database() in Web Worker freezing

[kingrongH]

What happend

adding const db = new Database('test.db') in worker.ts makes process freezing

Environment info

deno 1.40.5 (release, x86_64-apple-darwin)
v8 12.1.285.27
typescript 5.3.3

sqlite version 3.42.0
Macbook Pro 2020

Minimal example

Here is the minimal reproducible example

main.ts

const worker = new Worker(import.meta.resolve("./worker.ts"), {
  type: "module"
});


// Learn more at https://deno.land/manual/examples/module_metadata#concepts
if (import.meta.main) {
  worker.postMessage({
    msg: "msg1"
  });
  console.log("msg posted");
}

worker.ts

import { Database } from "jsr:@db/sqlite@0.11";
const thisWorker = self as unknown as Worker;
const db = new Database("test.db");

self.onmessage = (e) => {
    const { msg } = e.data;
    console.log(`received ${msg}`);
    thisWorker.postMessage({
        msg: `callback for ${msg}`
    });
    thisWorker.close();
  };

run code with the following command

$ deno run -A --unstable-ffi main.ts

then only msg posted is shown on the screen, and process just hang, expecting shown received msg1 and exiting as usual.

[DjDeveloperr]

I think this might be because this module loads asynchronously, and by the time you postMessage to that Worker, module hasn't really evaluated by then. This is because the dlopen we use from plug module is async and we use top-level await to resolve it. Try posting a message from the worker itself sometime after the import statement for sqlite module, and only when that is received on main thread, you can start sending messages.

[kingrongH]

tks, I think I got it, closing this issue

1. after new Worker('worker.ts') being evaluated, worker.ts starts runing asynchronously.
2. but new Database() just takes time, onmessage hasn't been registered by the time.
3. main.ts starts to postMessage, since there is no onmessage registered in worker.ts, there wont be any reponding

[DjDeveloperr]

Note that ES module evaluation is asynchronous. Normally it just resolves quickly, but in cases where we use top-level await, it will not be asynchronous. That is why we have to await the dynamic imports. While it may not look asynchronous here,

import { Database } from "jsr:@db/sqlite@0.11";

But it is actually awaiting the module evaluation of @db/sqlite here, which in turn makes your worker.ts module asynchronous.

new Worker() is sync. It will not wait for your asynchronous modules to evaluate. You will have to add such logic yourself such as posting a message from the worker as I suggested before.

new Database() is sync. It does not take time. But it's that import itself which is taking time here actually. But you are right in finding that onmessage is not registered by the time.

[kingrongH]

thanks for your detailed explanation. I think the way that put listener on worker.ts's ready message in main.ts and await it(by new Promise()) may be flawed, cuz by the time onmessage is registered in main.ts, worker.ts may have already been evaluated, there won't be any ready message after that.

Here I have draw a chart showing my concerning

By chart

As what's showing in this chart,

• worker.ts being executed in another thread (Thread A), total evaluation time is b
• main.ts being executed in deno's Main Thread, the evaluation time between new Worker() and onmessage is a

As they being executed in separated thread in parallel, time a can be greater than time b, in this situation, the efforts done for waiting worker.ts ready are just wasted.

Flawed Code Example

Here is a flawed code example, I use sleep for simulating main thread's slowness.

main.ts

const sleep = (ms: number) => new Promise(r => setTimeout(r, ms));

const worker = new Worker(import.meta.resolve("./worker.ts"), {
  type: "module"
});

// simulate main thread slowness
await sleep(200);

const worker_ready = () => new Promise<void>((resolve, _reject) => {
  const handleEvnet = (e: MessageEvent) => {
    const { msg } = e.data;
    console.log(`main.ts received msg: ${msg}`);
    if (msg === "ready") {
      resolve();
      worker.removeEventListener("message", handleEvnet);
    }
  }
  worker.addEventListener("message", handleEvnet);
});

// make sure worker ready
await worker_ready();
// then post some msg to worker
worker.postMessage({ msg: "main.ts msg"});

worker.ts

const thisWorker = self as unknown as Worker;
thisWorker.onmessage = e => {
    const { msg } = e.data;
    console.log(`worker.ts received msg: ${msg}`);
    thisWorker.close();
}

thisWorker.postMessage({ msg: "ready"});

Conclusion

As what's showing above, the effectiveness of awaiting worker's ready msg in main thread, relies on the gap of evaluation time, which may be flawed.

There should be a way that we can register initial ready listener for new Worker(), or maybe I missing something important, the conclusion is just wrong, lol.

EDIT: Learnt from The Basics of Web Workers, worker only starts by calling the postMessage()