A LIFO(Last-In-First-Out) job queue to provide back pressure and load shedding, while executing a queue of jobs
npm install --save jobqueue
or with yarn
as
yarn add jobqueue
options
:maxConcurrency: number
The number of concurrent jobs to execute at a timetimeout: number
the time after which the job should be cancelled and removed from the queuemaxJobs: number
the max number of jobs to be added to the queue for processing
Execute the set of jobs in the JobQueue
Wait for the job to be executed, based on the concurrency of the JobQueue
const JobQueue = require("../dist/index").default;
const { JobQueueFullError, JobTimeoutError } = require("../dist/index");
const dummyFetch = () =>
new Promise(resolve => {
setTimeout(() => resolve("success"), 1000);
});
let cancelledJobs = 0;
let timedOutJobs = 0;
// Initialize the job queue
const queue = new JobQueue({});
const jobs = Array.from({ length: 2 }).map(() => dummyFetch);
jobs.forEach(async job => {
const err = await queue.execute(job);
if (err instanceof JobQueueFullError) {
cancelledJobs += 1;
}
if (err instanceof JobTimeoutError) {
timedOutJobs += 1;
}
});
console.log(">>>>>>Timed out jobs", timedOutJobs);
console.log(">>>>>>Cancelled jobs", cancelledJobs);
For more examples refer the examples folder. To run the examples:
yarn build && node examples/queue-with-one-job.js
This is inspired by Dropbox's Bandaid proxy. Thanks a lot to Arpad for the Golang
implementation of JobQueue which was helpful in implementing this for JavaScript