Processors with multiple processes inconsistently process jobs

[Lemas97]

Is there an existing issue for this?

• I have searched the existing issues

Current behavior

When I set more than one processes on a processor,@OnQueueError event, I get the following Redis error (on every second aprox.):

Error: Command timed out at Timeout._onTimeout (...\node_modules\ioredis\built\Command.js:193:33) at listOnTimeout (node:internal/timers:569:17) at processTimers (node:internal/timers:512:7)

for this command brpoplpush [ 'bull:queueName:wait', 'bull:queueName:active', '5' ]

Despite the error spam. I see that jobs are not being processed correctly. Although they are moved to the “active” jobs, on Redis, they don't get processed, and they eventually stall, either forever either they get processed after some “stalls”.
Unfortunately this is not consistent, some jobs are being processed asap, some stall for a bit, and some stall “forever”. Restarting the NestJS server might process them.

We have observed that the more process functions you have on a queue, more jobs will be stalled and not processed.

The reason I need more than one process, is to handle different job types in the same queue.

Minimum reproduction code

https://github.com/Lemas97/nestjs-bull-issue

Steps to reproduce

1. yarn
2. yarn start:dev

The project consists of a NestJS project with 2 @processors. The first one contains only one @process (singleProcess.consumer.ts) and the second contains 9 @process functions (multipleProcess.consumer.ts)

On start, the project will automatically add jobs into both queues to reproduce the issue.

Result:

1. Jobs on singleProcess queue will run normally
2. Jobs on multiProcess queue will not start (some of them) and get stalled.
3. Errors will start appearing as the one mentioned above.

Expected behavior

The multiProcess queue should consistently process jobs.

Package version

0.6.1

Bull version

4.10.0

NestJS version

9.4.2

Node.js version

No response

In which operating systems have you tested?

• macOS
• Windows
• Linux

Other

No response

[kamilmysliwiec]

Please, report this issue in the bull repository