Best practice for handling concurrent jobs in a multi-instance Express app

[kbzowski]

I am currently using Nginx Unit to run multiple instances of an Express application. I have several cron jobs that perform database operations (using node-cron). These jobs need to be managed in such a way that they do not run simultaneously to avoid conflicts or data integrity issues. I am looking for the best practices or solutions to ensure that cron jobs are handled by only one process at a time.

Is there a way to designate an instance of the application under nginx unit to handle specific task (can I pass process id or some other value dynamically to identify first/main process)?

The only alternative I can see is to develop some kind of database locking mechanism that only allows one instance to do the task.

[tippexs]

Hi @kbzowski
sorry for the late response but I have questions.

Are we talking about the use of something like

var cron = require('node-cron');

cron.schedule('1-5 * * * *', () => {
  console.log('running every minute to 1 from 5');
});

inside of your express application?

If so the trigger for the cron comes from inside the application. Even if you have multiple processes of this application instance it should be quite easy to have a lock for crons by introducing lock-files, db or keyvalue-store entries.

I would check if a given task / cron is currently locked as a first action in my cron.schedule body.

Another option is to have a deticated application instance just for your crons, but this might not work at scale.