addBulk does not work with repeatable jobs

[jtassin]

Description

addBulk function does not work properly for repeatable jobs, the nextRepeatableJob is not invoked for repeatable jobs.

Minimal, Working Test code to reproduce the issue.

(An easy to reproduce test case will dramatically decrease the resolution time.)

The following code :

imgQueue.addBulk([{ data: { img }, opts: {repeatable: { every: 5_000 }}}])

is different than :

imgQueue.add({ img }, {repeatable: { every: 5_000 }})

In the add code we have this behaviour once the job is created :

if (opts.repeat) {
  return this.isReady().then(() => {
    return this.nextRepeatableJob(name, data, opts, true);
  });
} 

It is missing in the addBulk function.

Solutions

I'm OK to fix it. Here are two possible solutions :

• forbid addBulk for repeatable jobs it could sounds weird to bulk add repeatable jobs, but it is kind of breaking
• call nextRepeatableJob for each repeatable jobs of the batch once created. Bad for perfs but still better than multiple adds

which one do you prefer ?

Bull version

3.14

Additional information

Thx for your lib :)

[DmitryOlkhovoi]

Looks like similar bug #1761
Does .add work for you?

[epechuzal]

I also want to add that addBulk does not work with jobs that repeat with a cron expression, e.g.

const jobs = [
  { name: 'cron_job_hourly', opts: { repeat: { cron: '0 * * * *' } } },
  { name: 'cron_job_daily', opts: { repeat: { cron: '0 0 * * *' } } },
]
queue.addBulk(jobs);

If you have code like this, then two unexpected issues will happen:

• the jobs will execute immediately rather than wait until the designated cron time
• the jobs will also never repeat, as mentioned by @jtassin above

Solution would be to either add support for bulk adding repeatable jobs to a queue, or disable the feature entirely for repeatables to avoid confusion.

[manast]

addBulk does not support repeatable jobs, and I do not think it is worth implementing. The idea of addBulk is to remove overhead when adding huge amounts of jobs, it does not make so much sense for repeatable jobs which usually are not so many.

[epechuzal]

Makes sense to me - might be worth adding some kind of type check to stop jobs being added like this, or at least adding to declaration or documentation that it is not supported

[jtassin]

Makes sense to me - might be worth adding some kind of type check to stop jobs being added like this, or at least adding to declaration or documentation that it is not supported

+1 for type check and documentation

Looks like similar bug #1761
Does .add work for you?

Yes, .add works perfectly in my case

[manast]

fixed with taskforcesh/bullmq#544

[ebrahimmfadae]

addBulk does not support repeatable jobs, and I do not think it is worth implementing. The idea of addBulk is to remove overhead when adding huge amounts of jobs, it does not make so much sense for repeatable jobs which usually are not so many.

It is not just about performance, sometimes we need atomicity.