Jobs not running on time when scheduled sooner than the processEvery() cadence

[edwin-jones]

Hi there,

I'm trying to upgrade to Agenda 2.0.0 and I noticed a bug that doesn't appear to happen in 1.0.3.

If you have an agenda instance set to process every minute or so, but schedule a job to run in 5 seconds it doesn't actually run until the next time agenda checks the DB (i.e. the next minute or so). This means you either have to up how frequently you make requests to the DB or lose the ability to run jobs at a higher frequency than the timer you have set.

In 1.0.3 if a job was scheduled to run before the next job retrieval it would persist in memory and run, then update the job in the db before the next _findAndLockNextJob call.

[bonesoul]

same here

[shashvat-kedia]

@simison I would like to work on this

[simison]

@SD1998 PR's welcome!

[shashvat-kedia]

@simison I am new to this project could you help me to get started in the right direction? Thanks.

[simison]

README.md and especially debugging section should be useful: https://github.com/agenda/agenda#debugging-issues

At the top of that page, there's also Slack-link, where you can ask questions. :-)

[jaschaio]

Any update on this? I am encountering the same issue.

A bit of background: I am listening to events and while only one event is emitted due to how nodes custer mode works various workers can react to the same event. So I use agenda .unique() to make sure that only a single event gets processed:

    async onChange( change ) {

        // Create a new job
        var job = agenda.create( 'processDatabaseEvent', { change } );

        // Get idempotencyKey from the change event
        var { _id: { _data: idempotencyKey } } = change;

        // Make sure the job only runs once for this event
        job.unique( { idempotencyKey }, { insertOnly: true } );

        // Schedule to run now
        job.schedule( new Date() );

        // Save Job
        await job.save();

    }

But if the processEvery option is set to lets say the default 5 seconds in the worst case there is 5 seconds between scheduling the job and actually running it.

agenda.now() seems to execute a job immediately bypassing processEvery but unfortunately I can't use unique() in that case which means that I would run multiple jobs for a single event in node cluster mode.