BullMQ causes segmentation fault on NodeJS 18.12.0

[infinityfye]

NodeJS version: 18.12.0 LTS (docker image: lts-hydrogen)
BullMQ version: 2.3.1
REDIS Version: 6.2.6

Running on AWS ECS with Fargate launch type 2x(.5CPU, and 1GB Memory)

A simple express app. On running I get the following error:

IMPORTANT! Eviction policy is volatile-lru. It should be "noeviction"
IMPORTANT! Eviction policy is volatile-lru. It should be "noeviction"
IMPORTANT! Eviction policy is volatile-lru. It should be "noeviction"
IMPORTANT! Eviction policy is volatile-lru. It should be "noeviction"
IMPORTANT! Eviction policy is volatile-lru. It should be "noeviction"

Afterwhich, CPU and memory utilization spikes to 100% and I get:

segmentation fault (core dump)

[manast]

This information is not enough for us to pinpoint any issue in the library. We will need some code to reproduce it.

[infinityfye]

Okay, I'm really limited in what I can produce.

It's a FeathersJS/Express app.

We inject three queues and three workers in the app instance on bootstrap, following is an example of a queue where all three are exactly the same except for the names.

distance-queue.js

const { Queue } = require('bullmq');

// This sets up a queue where the worker does some haversine calculations
module.exports = function setupDistanceQueue(app) {
  const distanceQueue = new Queue(app.get('queues').distanceQueue, {
    connection: {
      host: app.get('redisQueue').host,
      port: Number.parseInt(app.get('redisQueue').port, 10),
    },
    defaultJobOptions: {
      removeOnComplete: true,
      attempts: 3,
      backoff: { type: 'exponential', delay: 60000 },
    },
  });

  app.set('distanceQueue', distanceQueue);
};

This is when the message MPORTANT! Eviction policy is volatile-lru. It should be "noeviction" is logged.

This only started happening when we moved to NodeJS 18.12.0.

[manast]

By definition, a segmentation fault is a bug in NodeJS. Now, there may be something in BullMQ that triggers that bug, I don't know, it could also be something with the rest of your code. If you can isolate it into some test case that ONLY depends on BullMQ, we can take a look into it.

The warning you are getting anyway is not the cause of any segmentation fault, it is just informative, and you really should fix it to avoid other issues with BullMQ down the road.

[infinityfye]

The same exact code runs on the same exact setup but for NodeJS 16 without generating that message or causing any failures, segmentation or otherwise. Literally the only change was the NodeJS image tag from 16 to 18.

Since nothing else changed, I pretty much assumed BullMQ was culprit because of this message. I don't think I'm particularly wrong either but I do have to debug a bit deeper.

Thank you @manast

[AfrazHussain]

@infinityfye Did you ever figure out the solution to this? I've just come across the exact same issue while trying to use BullMQ with AWS ElasticCache.

[infinityfye]

Yeah, the segfault was caused, as is the case usually, by a library we maintained that made heavy use of native code. In our issue, the segfault was caused because we changed to node 18 which uses openssl 3, while we were statically linking to openssl 1.1. I believe this issue is happening with several "native" database drivers as well, like pg-native.

I believe the first step you need to take @AfrazHussain is to figure out what exactly is causing the segfault. It's easy to write your own but here is a pretty popular one from npm: node-segfault-handler