Meteor Jobs

(inspired heavily by msavin:sjobs)

Run scheduled tasks with the simple jobs queue made just for Meteor. With tight MongoDB integration, this package is quick, reliable and effortless to use.

This is the async line of the package, for Meteor 2.8.1 and later including Meteor 3. For Meteor 1.3 to 2.x with the synchronous (Fibers) API, use wildhart:jobs-fibers instead; see Which package do I want? below.

• Jobs run on one server at a time
• Jobs run predictably and consecutively
• Job timers are super-efficient
• Jobs are stored in MongoDB
• No third party dependencies

It can run hundreds of jobs in seconds with minimal CPU impact, making it a reasonable choice for many applications. To get started, check out the quick start guide and the full API documentation below.

This repository publishes two packages from one codebase:

Both lines get the same features, with matching minor version numbers (2.2.x here is at feature parity with 1.2.x of wildhart:jobs-fibers).

• Meteor 3.0+: use wildhart:jobs 2.x. Nothing else runs on Meteor 3.
• Meteor 2.8 - 2.x: either works. wildhart:jobs 2.x uses the async MongoDB API introduced in Meteor 2.8, provided all your calls are awaited as per the migration guide; un-awaited fiber-style code misbehaves silently, since a returned Promise is always truthy. This lets you migrate your app to async/await while still on Meteor 2.x, before bumping to Meteor 3. If you would rather keep synchronous job functions for now, use wildhart:jobs-fibers.
• Meteor < 2.8: use wildhart:jobs-fibers.

Version 2.0.0 is async-only: every API method returns a Promise which must be awaited, including the this.* methods inside your job functions - see the migration guide. Apps on wildhart:jobs 1.0.x which are not ready for that should switch to wildhart:jobs-fibers, which continues the synchronous line with the same features: meteor remove wildhart:jobs, meteor add wildhart:jobs-fibers, and change the import path. The exported globals, API and jobs_data collection are unchanged.

This package has an API inspired by msavin:sjobs and in some cases can be a drop-in replacement. If you're coming from msavin:jobs read about the potentially breaking API differences. If any of these differences make this package unsuitable for you, please let me know and I'll consider fixing.

The main difference in this package compared to msavin:jobs is that this package doesn't continuously poll the job queue. Instead, it intelligently sets a single timer for the next due job. This means that most of the time this package is doing absolutely nothing, compared to msavin:jobs which can use significant CPU even when idle. It also means that jobs are executed closer to their due date, instead of potentially late due to the polling interval.

Unfortunately I found the job queue system in msavin:jobs too fundamentally built-in to modify and create a PR, so it was easier to write my own package.

First, install the package, and import if necessary:

meteor add wildhart:jobs

import { Jobs } from 'meteor/wildhart:jobs'

Then, write your background jobs like you would write your methods:

Jobs.register({
    "sendReminder": function (to, message) {
        var call = HTTP.put("http://www.magic.com/sendEmail", {
            to: to,
            message: message
        });

        if (call.statusCode === 200) {
            this.success(call.result);
        } else {
            this.reschedule({in: {minutes: 5}});
        }
    }
});

Finally, schedule a background job like you would call a method:

Jobs.run("sendReminder", "jon@example.com", "The future is here!");

One more thing: the function above will schedule the job to run on the moment that the function was called, however, you can delay it by passing in a special configuration object at the end:

Jobs.run("sendReminder", "jon@example.com", "The future is here!", {
    in: {
        days: 3,
    },
    on: {
        hour: 9,
        minute: 42
    },
    priority: 9999999999
});

The configuration object supports date, in, on, and priority, all of which are completely optional, see Jobs.run.

Version 2.0.0 introduces breaking changes to support Meteor 3.0's async database operations. All major API methods now return Promises and must be awaited. The same async API also runs on Meteor 2.8 - 2.x (which introduced the async MongoDB methods), so you can migrate your app to async/await before upgrading to Meteor 3.

All API methods are now async and return Promises:

// OLD (v1.x):
const jobDoc = Jobs.run("sendEmail", email, message);
Jobs.remove(jobDoc);
const count = Jobs.count("sendEmail");

// NEW (v2.0+):
const jobDoc = await Jobs.run("sendEmail", email, message);
await Jobs.remove(jobDoc);
const count = await Jobs.count("sendEmail");

Job context methods inside job functions are now async:

// OLD (v1.x):
Jobs.register({
    sendEmail: function(to, message) {
        sendEmail(to, message);
        this.success();  // or this.remove(), this.reschedule(), etc.
    }
});

// NEW (v2.0+):
Jobs.register({
    sendEmail: async function(to, message) {
        await sendEmail(to, message);
        await this.success();  // Must await all context methods
    }
});

All Jobs API methods requiring await:

• Jobs.run() - Schedule a job
• Jobs.execute() - Execute a job immediately
• Jobs.remove() - Remove a job
• Jobs.clear() - Clear jobs
• Jobs.replicate() - Replicate a job
• Jobs.reschedule() - Reschedule a job
• Jobs.findOne() - Find a job
• Jobs.count() - Count jobs
• Jobs.countPending() - Count pending jobs
• Jobs.start() - Start job queues
• Jobs.stop() - Stop job queues

TypedJob API is also fully async:

// NEW (v2.0+):
await sendReminderJob.withArgs('jon@example.com', 'Hello').run({in: {days: 1}});
await sendReminderJob.clear('*', 'arg1');
const count = await sendReminderJob.count('arg1');

Your job functions can now be async and use await:

Jobs.register({
    async processPayment(userId, amount) {
        const user = await Users.findOneAsync(userId);
        const result = await stripe.charges.create({...});

        if (result.success) {
            await this.success();
        } else {
            await this.reschedule({in: {minutes: 5}});
        }
    }
});

With version 1.0.18 we introduced a more convenient and strongly typed wrapper Class around our traditional API. You can still use the old API, and even upgrade to this new version with no additional work, then you are free to gradually update your code to the new API.

** Don't forget to copy our new wildhart-jobs.d.ts into your project's @types folder.

Benefits of the new API:

• All job parameters are strongly typed, so in code which schedules a job you will get IDE warnings if the types are incorrect.
• No more scheduling jobs by string name, so no risk of typos.

With the new API, the above code would be replaced with:

import { TypedJob } from "meteor/wildhart:jobs";

export const sendReminderJob = new TypedJob('sendReminders', function(to: string, message: string) {
	...
});

Note that when defining the job, that's only only place you need to refer to the job with a string name.

When scheduling the job, you can reference the class instance directly:

import { sendReminderJob } from './reminders';

sendReminderJob.withArgs('jon@example.com", The future is here!').run({
    in: {
        days: 3,
    },
    on: {
        hour: 9,
        minute: 42
    },
    priority: 9999999999
});

Almost all of the traditional API can be replaced with this new API:

// as example above
sendReminderJob.withArgs(...).run(configObject);
// equivalent to Jobs.clear('*', 'sendReminder', '*', ...args);
sendReminderJob.clear('*', ...args);
// NEW API equivalent to Jobs.collection.clear({...query, name: 'sendReminderJob');
sendReminderJob.clearQuery(query);

// same as Jobs.remove(....), but without having to import "Jobs"
const scheduledJob: JobDocument | false = myJob.withArgs(...).run(...);
sendReminderJob.remove(scheduledJob);
// or
sendReminderJob.remove(scheduledJob._id);

// equivalent to Jobs.start('sendReminders');
sendReminderJob.start();
// equivalent to Jobs.stop('sendReminders');
sendReminderJob.stop();
// equivalent to Jobs.count('sendReminders', 'jon@example.com');
sendReminderJob.count('jon@example.com');
// equivalent to Jobs.findOne('sendReminders', 'jon@example.com');
sendReminderJob.findOne('jon@example.com');
// this is new API equivalent to Jobs.update({query, ..name: 'sendReminderJob'}, options);
sendReminderJob.update(query, options);

// if you need to query the Jobs collection directly, the original name of the job can be obtained:
sendReminderJob.name; // == 'sendReminders'

Further details of these methods are as per the traditional API below.

One big caveat of the new API is that to run a job you have to import the code from the file where the job was defined, which by definition should be exposed on the server side only. Therefore, in shared client/server code (e.g. a Meteor Method) if you are used to doing:

if (Meteor.isServer) {
	Jobs.run('sendEmail', 'jon@example.com', 'hello', {in: {days: 1}});
}

You have to be careful not to import the server-side code into the front-end, by using import().then():

if (Meteor.isServer) {
	import('../../server/reminderJobs').then(({sendEmailJob}) => {
		sendEmailJob.withArgs(...).run(...));
	});
}

Jobs.register() and Jobs.run() are all you need to get started, but that's only the beginning of what the package can do. To explore the rest of the functionality, jump into the documentation:

• Jobs.configure
• Jobs.register
• Jobs.run
• Jobs.execute
• Jobs.reschedule
• Jobs.replicate
• Jobs.start
• Jobs.stop
• Jobs.clear
• Jobs.remove
• Jobs.jobs
• Jobs.collection
• Repeating Jobs
• Async Jobs/Promises
• Crash recovery
• Monti APM
• Bulk Operations
• Version History

Jobs.configure() allows you to configure how the package should work. You can configure one option or all of them. Defaults are shown in the code below:

Jobs.configure({
    // (milliseconds) specify how long the server could be inactive before another server
    // takes on the master role (default = 5min)
    maxWait: Number,

    // (milliseconds) specify how long after server startup the package should start running
    startupDelay: Number,

    // determine how to set the serverId - see below. (default = random string)
    setServerId: String || Function,

    // this server never runs the job queue (default = false) - see "Dedicated jobs server" below.
    dontRunJobs: Boolean,

    // determine if/how to log the package outputs (default = console.log)
    log: Boolean || Function,

    // specify if all job queues should start automatically on first launch (default = true)...
    //  ... after server relaunch the list of paused queues is restored from the database.
    autoStart: Boolean,

    // whether to mark successful just as successful, or remove them,
    // otherwise you have to resolve every job with this.success() or this.remove().
    // Pass null to return to the default.
    defaultCompletion: 'success' | 'remove' | null,

    // requeue jobs left 'executing' whenever a server takes control of the queue (default = false).
    // Makes execution at-least-once, see "Crash recovery" below before enabling.
    requeueOnTakeover: Boolean,

    // (milliseconds) requeue jobs which have been 'executing' for longer than this,
    // checked on every ping. 0 (default) = off. See "Crash recovery" below.
    maxExecutionTime: Number,

    // Monti APM jobs dashboard integration (default = false). true traces every job run,
    // {pendingInterval: ms} additionally reports pending counts. See "Monti APM" below.
    monti: Boolean || {pendingInterval: Number},
})

setServerId - In a multi-server deployment, jobs are only executed on one server. Each server should have a unique ID so that it knows if it is control of the job queue or not. You can provide a function which returns a serverId from somewhere (e.g. from an environment variable) or just use the default of a random string. In a single-server deployment set this to a static string so that the server knows that it is always in control and can take control more quickly after a reboot.

By default any server may take control of the job queue, so with several servers running the same code you cannot choose which one runs the jobs. To keep them on one or more dedicated servers (for example a box which handles no user connections, see #30), set dontRunJobs: true on every other server, typically from an environment variable:

Jobs.configure({
    dontRunJobs: !process.env.JOB_RUNNER,  // only the dedicated server(s) have JOB_RUNNER set
    setServerId: process.env.JOB_RUNNER,   // optional: a static id lets a dedicated server resume control instantly after a restart
});

• A server with dontRunJobs never takes control of the queue, even when the server in control goes quiet. If every dedicated server is down, jobs wait.
• Several dedicated servers elect among themselves as usual: whichever takes control first runs the jobs and the others take over after maxWait if it goes quiet. Each one still needs its own unique setServerId (or the random default).
• A server with dontRunJobs can still schedule jobs with Jobs.run(), pause and resume the queue with Jobs.stop() / Jobs.start(), and run a pending job on demand with Jobs.execute(), which runs the job function on the server which calls it.
• The first deployment which introduces the option can leave the queue idle for up to maxWait if an older server was in control, until its last ping goes stale. After that a dedicated server with a static setServerId resumes control instantly when it restarts.

If two servers are ever started with the same setServerId, both believe they are in control and every job runs twice. Since 2.2.0 the package detects this and logs a warning on each of them.

Jobs.register() allows you to register a function for a job.

Jobs.register({
	sendEmail: function (to, content) {
		var send = Magic.sendEmail(to, content);
		if (send) {
			this.success();
		} else {
			this.reschedule({in: {minutes: 5}});
		}
	},
	sendReminder: function (userId, content) {
		var doc = Reminders.insert({
			to: userId,
			content: content
		})

		if (doc) {
			this.remove();
		} else {
			this.reschedule({in: {minutes: 5}});
		}
	}
});

// or NEW API:
const sendEmail = new TypedJob('sendEmail', function(to: string, content: EmailDoc) {
	...
});
const sendReminder = new TypedJob('sendReminder', function(to: string, content: ReminderContent) {
	...
});

Each job is bound with a set of functions to give you maximum control over how the job runs:

• this.document - access the job document
• this.success() - tell the queue the job is completed
• this.failure() - tell the queue the job failed
• this.reschedule(config) - tell the queue to schedule the job for a future date
• this.remove() - remove the job from the queue
• this.replicate(config) - create a copy of the job with a different due date provided by config (returns the new jobId)

Each job must be resolved with success, failure, reschedule, and/or remove.

See Repeating Jobs and Async Jobs/Promises

Jobs.run() allows you to schedule a job to run. You call it just like you would call a method, by specifying the job name and its arguments. At the end, you can pass in a special configuration object. Otherwise, it will be scheduled to run as soon as possible.

var jobDoc = Jobs.run("sendReminder", "jon@example.com", "The future is here!", {
    in: {
        days: 3,
    },
    on: {
        hour: 9,
        minute: 42
    },
    priority: 9999999999,
    singular: true
});

// or NEW API:
sendReminderJob.withArgs("jon@example.com", "The future is here!").run(...);

Jobs.run() returns a jobDoc.

The configuration object supports the following inputs:

• in - Object
  • The in parameter will schedule the job at a later time, using the current time and your inputs to calculate the due time.
• on - Object
  • The on parameter override the current time with your inputs.
• in and on - Object
  • The supported fields for in and on can be used in singular and/or plural versions:
    • millisecond, second, minute, hour, day, month, and year
    • milliseconds, seconds, minutes, hours, days, months, and years
  • The date object will be updated in the order that is specified. This means that if it is year 2017, and you set in one year, but on 2019, the year 2019 will be the final result. However, if you set on 2019 and in one year, then the year 2020 will be the final result.
• priority - Number
  • The default priority for each job is 0
  • If you set it to a positive integer, it will run ahead of other jobs.
  • If you set it to a negative integer, it will only run after all the zero or positive jobs have completed.
• date - Date
  • Provide your own date. This stacks with the in and on operator, and will be applied before they perform their operations.

• unique - Boolean
  • If a job is marked as unique, it will only be scheduled if no other job exists with the same arguments
• singular - Boolean
  • If a job is marked as singular, it will only be scheduled if no other job is pending with the same arguments
• awaitAsync - Boolean
  • If an async job with run with awaitAsync: true is running, then no other job of the same name will start until the running job has completed.
• jobId - String
  • Use your own _id for the job document instead of a generated one. This makes scheduling idempotent: if a job with that id already exists (in any state), Jobs.run() logs Job with this id already exists, calls the callback with that error, and returns false, exactly like unique and singular. Useful when the id is derived from your own data (e.g. "reminder-" + orderId) so you can Jobs.remove(id) or Jobs.reschedule(id, ...) later without querying.
  • "In any state" includes finished jobs: a job resolved with this.success() (or defaultCompletion: 'success') keeps its id occupied until it is removed, so pair jobId with this.remove() or defaultCompletion: 'remove' if you want to schedule the same id again later. Jobs.replicate() of a job with a custom id gives the copy a generated id.
  • Must be a non-empty string. Note that a trailing argument object which happens to contain a jobId key is now treated as the configuration object, as with every other configuration key.
• retries - Number
  • How many times to run the job again if its function throws (or its promise rejects). The default is 0: the job is marked 'failure' on the first error. With retries: 2 the job runs up to 3 times. An explicit this.failure() is never retried. Must be a non-negative integer.
• retryIn - Object
  • How long to wait before each retry, in the same format as in (e.g. {minutes: 5}). The default is to retry as soon as possible.
  • The job document records how many times the current scheduling of the job has run in attempts, also available in the job function as this.document.attempts. attempts is recorded for every job, not only those with retries.
  • Rescheduling a job with this.reschedule() or Jobs.reschedule() starts a new run cycle and resets attempts, so a repeating job gets its full retries on every run. Jobs.replicate() copies retries and retryIn to the new job but not attempts.

• callback - Function
  • Run a callback function after scheduling the job

Jobs.execute() allows you to run a job ahead of its due date. It can only work on jobs that have not been resolved.

Jobs.execute(doc) // or (doc._id)
// or NEW API
sendReminderJob.execute(doc); // or (doc._id)

For an async job the returned promise resolves as soon as the job function has been started, like the queue itself does. Pass {awaitCompletion: true} to resolve only once the job function has finished and the job's state has been resolved, which is what you usually want when executing a job from a method or a test:

await Jobs.execute(doc, {awaitCompletion: true});

The promise resolves to how the job was resolved: 'success', 'failure', 'reschedule' or 'remove' (including a resolution applied by defaultCompletion). For an async job without awaitCompletion it resolves to 'executing', since the job is still running. It resolves to false if the job was not found or is not pending.

const result = await Jobs.execute(doc, {awaitCompletion: true});
if (result == 'failure') { /* ... */ }

Jobs.reschedule() allows you to reschedule a job. It can only work on jobs that have not been resolved.

Jobs.reschedule(job, { // or (job._id)
	in: {
		minutes: 5
	},
	priority: 9999999
});
// or NEW API
sendReminderJob.execute(job, {...}); // or (job._id, {...});

The configuration is passed in as the second argument, and it supports the same inputs as Jobs.run().

Jobs.replicate() allows you to replicate a job.

var jobId = Jobs.replicate(job, { // or (job._id, {...
	in: {
		minutes: 5
	}
})
// or NEW API
sendReminderJob.execute(job, {...}); // or (job._id, {...});

Jobs.replicate() returns a jobId.

Jobs.start() allows you start all the queues. This runs automatically unless autoStart is set to false. If you call the function with no arguments, it will start all the queues. If you pass in a String, it will start a queue with that name. If you pass in an Array, it will start all the queues named in the array.

// Start all the queues
Jobs.start()

// Start just one queue
Jobs.start("sendReminder")
// or NEW API
sendReminderJob.start();

// Start multiple queues
Jobs.start(["sendReminder", "sendEmail"])

Unlike msavin:sjobs, this function can be called on any server and whichever server is currently in control of the job queue will be notified.

Jobs.stop() allows you stop all the queues. If you call the function with no arguments, it will stop all the queues. If you pass in a String, it will stop a queue with that name. If you pass in an Array, it will stop all the queues named in the array.

// Stop all the queues
Jobs.stop()

// Stop just one queue
Jobs.stop("sendReminder")
// or NEW API
sendReminderJob.stop();

// Stop multiple queues
Jobs.stop(["sendReminder", "sendEmail"])

Unlike msavin:sjobs, this function can be called on any server and whichever server is currently in control of the job queue will be notified.

If you need to stop all jobs via mongo use:

mongo> db.jobs_dominator_3.update({_id:"dominatorId"}, {$set: {pausedJobs: ['*']}});

The in-control server should observe the change and stop instantly. Use {$unset: {pausedJobs: 1}} or {$set: {pausedJobs: []}} to start all the queues again.

Jobs.clear() allows you to clear all or some of the jobs in your database.

var count = Jobs.clear(state, jobName, ...arguments, callback);
e.g:
count = Jobs.clear(); 		// remove all completed jobs (success or failure)
count = Jobs.clear('*');	// remove all jobs
count = Jobs.clear('failure', 'sendEmail', 'jon@example.com', function(err, count) {console.log(err, count)});
// or NEW API
count = sendEmailJob.clear('failure', 'jon@example.com', ...);

Parameters:

• state for selecting a job state (either pending, success, failure, or * to select all of them), or omit to all except pending jobs.
• jobName to only remove jobs with a specific name.
• provide arguments to match jobs only with the same arguments.
• callback to provide a callback function with error and result parameters, where result is the number of jobs removed.

Jobs.remove() allows you to remove a job from the collection.

var success = Jobs.remove(doc); // or (doc._id)
// or NEW API
sendEmailJob.remove(doc); // or (doc._id)

Jobs.jobs gives access to an object of defined job functions:

var jobNames = Object.keys(Jobs.jobs);  // ['sendEmail', 'sendReminder']
var nJobTypes = jobNames.length;        // 2

Jobs.collection allows you to access the MongoDB collection where the jobs are stored. Ideally, you should not require interaction with the database directly.

Repeating jobs can be created by using this.reschedule() in the job function, e.g.:

Jobs.register({
	processMonthlyPayments() {
		this.reschedule({in: {months: 1}});
		processPayments();
	},
});

Jobs.run('processMonthlyPayments', {singular: true});

Since this package doesn't keep a job history (compared with msavin:sjobs), you can use this.reschedule() indefinitely without polluting the jobs database, instead of having to use this.replicate() followed by this.remove().

The job function can use async/await or return a promise:

Jobs.register({
	async asyncJob(...args) {
		await new Promise(resolve => Meteor.setTimeout(() => resolve(0), 4000));
		this.remove();
	},
	promiseJob(...args) {
		return new Promise(resolve => Meteor.setTimeout(() => {
			this.remove();
			resolve(0);
		}, 8000));
	},
});

This defers the error message 'Job was not resolved with success, failure, reschedule or remove' until the promise resolves. Note that:

• While jobs are executing their status is set to 'executing'.
• Other jobs of the same type will still run when scheduled while asynchronous jobs are executing, unless the running job was configured with awaitSync: true, in which case the pending job will wait until the previous job of that name has completed.
• Asynchronous code may need to be wrapped in Meteor.bindEnvironment().

A job is marked 'executing' (with a startedAt date) just before its function runs. If the server in control of the queue crashes, is killed, or restarts while jobs are executing, those jobs stay 'executing' forever and never run again. By default the package does nothing about this: a lost job is lost, and no job ever runs twice (at-most-once).

Two opt-in settings in Jobs.configure() change that trade-off to at-least-once:

• requeueOnTakeover: true - whenever a server takes control of the queue (a fresh start, a restart of the server in control with a static setServerId, or a takeover after maxWait), every job still 'executing' is returned to 'pending' and runs again straight away. Since only the server in control executes jobs, such a job was normally started by a server which is gone.
• maxExecutionTime (milliseconds) - on every ping, the server in control requeues jobs which started more than this long ago. This covers a job function which hangs (for example on a network call with no timeout) while its server stays alive. Keep it comfortably longer than your longest job.

Before enabling either, make sure your job functions are safe to run more than once (idempotent, or checking your own data before acting). A requeued job runs its function again from the start, and the first run may have partly or fully completed:

• The old server may have been stalled rather than dead (a long GC pause, a database outage longer than maxWait, a synchronous job blocking the event loop) and will finish its copy of the job after the takeover.
• Jobs started with Jobs.execute() on a server which is not in control are also 'executing' and are requeued by a takeover while they run.
• A job which crashes the server itself (an uncaught exception outside the job's promise, running out of memory) is requeued on every restart and crashes the server again. Give such jobs a retries value: a requeued run counts as an attempt, and a job which has used all of its attempts is marked 'failure' instead of being requeued, so the loop ends.

You can also call Jobs.requeueExecuting() yourself, optionally with a Date to only requeue jobs started before it. It returns the number of jobs requeued. A single-server deployment can, for example, call it once from Meteor.startup() instead of enabling requeueOnTakeover.

If your app uses Monti APM (montiapm:agent 2.50 or later, or the Meteor 3 agent), the package can feed its Jobs dashboard. It is opt-in:

Jobs.configure({
    monti: true,                             // trace every job run, count jobs added by Jobs.run()
    // or
    monti: {pendingInterval: 20 * 1000},     // ... and report the number of pending jobs every 20s
});

montiapm:agent is not a dependency of this package. The agent is looked up at run time, so an app without it (or on any version of it) builds unchanged. If monti is set but the agent is not found, or is older than 2.50 (no jobs API), the package logs one warning and runs jobs without tracing.

What you get:

• Every job run is a trace named after the job (job names map 1:1 to Monti trace names, and Monti suggests keeping those to a few dozen). The trace's delay is now - due, i.e. how late the job started; it is 0 for a job run ahead of time with Jobs.execute(). The trace's start data contains the job _id, its arguments and the attempt number. If arguments are sensitive, mask them with Monti's own Monti.tracer.addFilter().
• A run is errored only when the job function throws or rejects. A job which calls this.failure() shows as a completed run. A retry (see retries in Jobs.run()) is a separate run with its own delay.
• Added counts every job inserted by Jobs.run(). Jobs refused by unique, singular or a duplicate jobId are not counted.
• Pending counts are off by default (Monti recommends it for performance reasons). With pendingInterval, the server in control of the queue reports the number of pending jobs for every registered job name, right after taking control and then every interval. Each report is one aggregation over the pending documents, which the package's {name, due, state} index cannot serve on its own; if your jobs_data collection is large, add a {state: 1, name: 1} index yourself before enabling it. Monti suggests an interval of 10 to 50 seconds.
• A job run with Jobs.execute() from inside a Meteor method (or any other Monti trace) is folded into that trace rather than shown as a job. This is how the agent behaves and is not configurable here.

The job queue intelligently prevents lots of a single job dominating the job queue, so feel free to use this package to safely schedule bulk operations, e.g, sending 1000s of emails. Although it may take some time to send all of these emails, any other jobs which are scheduled to run while they are being sent will still be run on time. Run each operation as its own job (e.g, 1000 separate "sendSingleEmail" jobs rather than a single "send1000Emails" job. The job queue will run all 1000 "sendSingleEmail" jobs in sequence, but after each job it will check if any other jobs need to run first.

If any of these differences make this package unsuitable for you, please let me know and I'll consider fixing.

• This package doesn't keep a job history.
• failed jobs are not retried, unless they have already been rescheduled.
• The Job configuration object doesn't support the data attribute - I never found any use for this.
• The following Jobs.configure() options are not available or different:
  • interval - this package doesn't regularly query the job queue for due jobs, instead it intelligently sets a timer for the next job.
  • getDate
  • disableDevelopmentMode
  • remoteCollection
  • autoStart - only relevant on first launch. On relaunch the list of paused queues is restored from the database.
• The following Jobs.configure() options have additional options:
  • setServerId can be a String as as well as a Function
  • log can be a Boolean as well as a Function
• In a job function, this.set() and this.get() are not provided - I never found any use for this.
• In a job function, this.success() and this.failure() to not take a result parameter - this package doesn't keep a job history
• singular jobs only check for pending jobs of the same name, so they can be run again even if a previous job failed.
• Jobs.start() and Jobs.stop() can be called on any server and whichever server is in control of the job queue will be notified.
• Jobs.cancel() doesn't exist. Just remove it with Jobs.remove() - I don't see the point in keeping old jobs lying around.
• Jobs.clear() can take additional argument parameters to only delete jobs matching those arguments.
• Jobs.jobs doesn't exist in msavin:sjobs

The package has a meteor test-packages suite in tests/. From the package directory, after npm install (dev dependencies only, nothing is shipped):

npm test            # run the suite once
npm run test:watch  # keep the test app running and re-run on file changes
npm run check       # type-check jobs.ts and TypedJob.ts with tsc

npm test runs:

TEST_CLIENT=0 meteor --release METEOR@3.5.2 test-packages ./ --port 3100 --once --driver-package meteortesting:mocha

TEST_CLIENT=0 skips the client run, which has nothing to test (the script uses cross-env so this works in any shell; on Windows cmd by hand it is set TEST_CLIENT=0 && meteor ...). Keep --port whenever another Meteor app is running on the default ports: the test app would otherwise share that app's MongoDB on port 3001 and the two would pick up each other's jobs.

• Detect old MontiAPM agent, show min required version is 2.50.

• Documentation only: the synchronous (Fibers) line for Meteor 1.3 to 2.x is now published as wildhart:jobs-fibers 1.2.0, at feature parity with this release. See "Which package do I want?"

• Opt-in Monti APM jobs dashboard integration: Jobs.configure({monti: true}) traces every job run and counts added jobs, {monti: {pendingInterval}} also reports pending counts. The agent is found at run time, so montiapm:agent is not a dependency. Requested in #32
• Jobs.configure({dontRunJobs: true}) keeps a server from ever running the job queue, so jobs can be kept on dedicated servers (see "Dedicated jobs server"). Requested in #30
• A warning is logged when two servers are running with the same setServerId (both would run every job)
• Fixed: Jobs.execute() could resolve before the state write of a sync job function's un-awaited this.success() (or failure/remove/reschedule) had reached the database

All new behaviour is opt-in; existing apps upgrade without change. Thanks to @harryadel for the features and the test suite.

• Jobs.execute(job, {awaitCompletion: true}) waits for an async job function to finish before resolving
• Jobs.execute() resolves to how the job was resolved ('success', 'failure', 'reschedule', 'remove'), 'executing' for an async job still running, or false if the job was not found or not pending
• Jobs.run() accepts a jobId config option to choose the job document's _id; a duplicate id returns false like unique/singular
• Jobs.run() accepts retries and retryIn to rerun a job whose function throws. Rescheduling a job starts a new run cycle, so repeating jobs get their full retries each time
• Crash recovery (opt-in, see "Crash recovery"): requeueOnTakeover requeues jobs left 'executing' by a crashed or restarted server when a server takes control; maxExecutionTime requeues jobs executing for too long; Jobs.requeueExecuting() does the same on demand
• New job document fields: attempts on every job (runs of the current scheduling, reset by a reschedule) and startedAt while a job is executing
• Note: a trailing argument object containing a jobId, retries or retryIn key is now recognised as the config object, as with every other config key
• Jobs.configure({defaultCompletion: null}) returns to the default
• Fixed: an overdue job produced a negative timer value (Node warned and clamped it to 1 ms)
• Package test suite (npm test, see "Running the tests") and JSDoc on the public types

• BREAKING CHANGE: Full migration to Meteor 3.0 async database operations
• All database operations now use async/await patterns
• Jobs.run(), Jobs.execute(), Jobs.remove(), Jobs.clear(), Jobs.replicate(), Jobs.reschedule(), Jobs.findOne(), Jobs.count(), Jobs.countPending(), Jobs.start(), and Jobs.stop() now return Promises
• Job context methods (this.success(), this.failure(), this.remove(), this.reschedule(), this.replicate()) are now async
• Replaced deprecated _ensureIndex() with createIndexAsync()
• All Mongo collection operations migrated to async methods (insertAsync, updateAsync, removeAsync, findOneAsync, countAsync, upsertAsync)
• TypedJob class methods now return Promises
• Updated TypeScript definitions for async methods
• Package version constraint updated to support Meteor 3.0 and TypeScript 5.0; minimum Meteor version is now 2.8.1 (the async MongoDB API is required)

• Added new strongly-typed API.

• Added awaitAsync: true option for async jobs. Fixes #14.

• Fixed not accepting startUpDelay in Jobs.Configure. Fixes #13.

• Removed typescript verison constraint.

• Added the 'defaultCompletion' Jobs.configure option. Suggested in #10.

• Better support for Async Jobs/Promises. Fixes #7.
• While jobs are executing their status is set to 'executing'.

• Capped timeout to 24 hours to avoid node limit. Fixes #5.

• Fix bug when using months to set the job time.
• Add return values to this.remove() etc within job.
• Fix observer/timeout race condition executing same job twice

• Prevent console.logging done jobs.

• Allow job queue to be paused while in executing loop.

• Fix bug when logging observer.

• Jobs which result in an error but have already been rescheduled will still run again at the rescheduled time.
• Access to the list of defined job types with Jobs.jobs.

• Can start/stop the job queue.

• Prevent a single job type from dominating the job queue (e.g. bulk email sending).

• First release.