addBulk with multiple repeating jobs only enqueues one repeating job

[higgins]

Hello!
First, thank you for this library. 🏆

Context

I'm initializing a queue with around 1000 repeatable jobs via the addBulk interface. A QueueScheduler for the queue has been instantiated. The jobs are initially enqueued and consumed just fine, but only the first job in the jobs array handed to addBulk repeats even though all jobs have the same repeat options.

Problem

I expect all jobs I pass to addBulk with a repeat option to repeat as configured.

Demo

  const MAIN_QUEUE = 'demo';
  const queue = new Queue(MAIN_QUEUE, { connection });
  const queueScheduler = new QueueScheduler(MAIN_QUEUE, { connection });
  const worker = new Worker(MAIN_QUEUE, async job => {
    console.log('Received a job!', job.data.name);
  }, { connection });
  ...
  await queue.addBulk(
    [
        {
          name,
          data: {name: 'job 1' },
          opts: {
            repeat: {
              every: 1000,
            }
          }
        },
        {
          name,
          data: {name: 'job 2' },
          opts: {
            repeat: {
              every: 1000,
            }
          }
        },
    ]
  )

# See the following logged:
// Received a job! job 1
// Received a job! job 2
// Received a job! job 1
// Received a job! job 1
// Received a job! job 1
...

version 1.16.1

[Stormtv]

Have you tried setting a custom name or jobId for each repeatable job?

https://docs.bullmq.io/guide/jobs/repeatable

Since repeatable jobs are delayed jobs, and the repetition is achieved by generating a new delayed job precisely before the current job starts processing. The jobs require unique ids which avoid duplicates, which implies that the standard jobId option does not work the same as with regular jobs. With repeatable jobs the jobId is used to generate the unique ids, for instance if you have 2 repeatable jobs with the same name and options you could use the jobId to have 2 different repeatable jobs:

import { Queue, QueueScheduler } from 'bullmq'

const myQueueScheduler = new QueueScheduler('Paint');
const myQueue = new Queue('Paint');

// Repeat job every 10 seconds but no more than 100 times
await myQueue.add('bird', { color: 'bird' }, 
  {
    repeat: {
      every: 10000,
      limit: 100
    },
    jobId: "colibri"
  });

await myQueue.add('bird', { color: 'bird' }, 
  {
    repeat: {
      every: 10000,
      limit: 100
    },
    jobId: "pingeon"
  });

[higgins]

Hey @Stormtv ! Yeah, we've tried adding jobId to each of the job objects we pass to addBulk but still the only one that repeats is the first job it receives.

[roggervalf]

I guess this issue is also related OptimalBits/bull#1731

[manast]

fixed with #544