How to use Worker with typescript processor file?

[alexandrsek]

Hello everyone! I was wondering, is it possible to use sandboxed jobs with typescript processor file?
I know its possible to use worker_threads with typescript by using require("ts-node").register().
Any ideas how to make it possible?

[manast]

You may have some luck using ts-node to run your workers. I am afraid, typescript+node+esm6 is basically broken in 2022. When the dust settles this situation may improve, but currently, it is really painful and I do not have any magical solutions to make this work in a satisfactory manner. I will write a blog post if I can find some workaround or some way to make this work as painless as possible.

[mrbrianevans]

Have you considered using native nodejs worker_threads instead of child_process in the implementation of this library? (not only for this issue, but perhaps other benefits as well)

[alexandrsek]

I was able to use typescript processors thanks to an example I saw in the original Bull library.
OptimalBits/bull#2150

You just need to switch from ts-node-dev to nodemon + ts-node.

@mrbrianevans thats an interesting suggestion.

My goal is to concurrently process multiple jobs without blocking the main event loop that handles API requests and file upload, thats why I was wondering...

1. Should I split queue and main app as separate Node.js apps?
2. Will there be any benefit from using pm2 cluster? Can cluster work with worker_threads?

[manast]

I wrote a blog post that hopefully clarifies a few things: https://blog.taskforce.sh/using-typescript-with-bullmq/

[manast]

Let me know if I need to complete the post with more information.

[alexandrsek]

Thats exactly what I was looking for, thank you!

[notflip]

@manast Thanks for the blog post, I've been having issues with nodemon and child_processes that hang, doesn't seem to happen with tsc-watch.

[manast]

You could also try using worker threads instead: https://api.docs.bullmq.io/interfaces/WorkerOptions.html#useWorkerThreads