Configure Kickq

Thanasis Polychronakis edited this page Feb 5, 2016 · 9 revisions

kickq.config(key, value)

The kickq.config() method acceps an Object with key value pairs or a key value pair.

// accepts an object with key/value pairs.
  redisNamespace: 'kickq'

// accepts a key/value pair.
kickq.config('redisNamespace', 'kickq');

// reset all config params, return to original state.

Configuration Options

Kickq Configuration

Option :: debug

Type: boolean Default: false

Will enable logging to console and log all messages generated.

Option :: purgeJobs

Type: boolean Default: true

Forever delete finished job items.

Option :: purgeTimeout

Type: number Default: 86400000 (milliseconds, default is 1 day)

When a job completes it is scheduled for purging. This timeout defines the time to wait before actually deleting the job record from redis.

Redis Configuration

Option :: redisPort

Type: number Default: 6389

Define a different port for redis.

Option :: redisHost

Type: string Default: ""

Define a different host for redis.

Option :: redisPassword

Type: string | null Default: null

Set a redis password if you have one.

Option :: redisOptions

Type: Object | null Default: null

More specific options for the redis client as defined in the redis package docs.

Option :: redisNamespace

Type: string Default: "kickq"

Define the master key namespace for redis, that kickq will use to store data.

Jobs Configuration

Option :: jobFlags

Type: Object Default: None

Set per job name options. Each key represents a job name.

You can use any of the options marked with ✓ per job option from the list bellow, options are applied cascadingly with the more specific overriding.


  retryCount: 2 // set global retry count value to 2

  jobFlags: {
    'job name': {
      // all options that allow "per job" configuration, e.g:
      retry: true,
      retryCount: 1 // override "retryCount" to 1 for the "job name" job.

All the following options can be set in the global context or per Job Name using the jobFlags option.

Option :: processTimeout

Type: number Default: 10000 (milliseconds) ✓ per job option

The global timeout for processing tasks, in milliseconds.

Option :: delay

Type: ?number (milliseconds) default null ✓ per job option

Using a delay will add the job to the scheduler for future execution as defined by the delay, expressed in milliseconds.

Option :: ghostRetry

Type: boolean default true ✓ per job option

A job gets ghosted when the process function does not invoke the callback or return a promise, triggering the processTimeout limit.

Option :: ghostTimes

Type: number default 1 ✓ per job option

How many times to retry processing a ghost job.

Option :: ghostInterval

Type: number default 1800000 (ms, half an hour) ✓ per job option

A job gets ghosted when the process function does not invoke the callback or return a promise, triggering the processTimeout limit.

Option :: hotjob

Type: boolean default false ✓ per job option

Tombstoning allows for run-time invocation of the new job. The job creator has a chance to wait for the completion of the new job within the timeout defined.

Option :: hotjobTimeout

Type: number default 10000 (milliseconds) ✓ per job option

Time to wait for a job to get processed before hotjob callbacks timeout, in milliseconds.

Option :: retry

Type: boolean default false ✓ per job option

Allow for a failed job to retry execution.

Option :: retryTimes

Type: number default 3 ✓ per job option

How many times to retry before finally giving up

Option :: retryInterval

Type: number default 180000 (ms, half an hour) ✓ per job option

How long to wait before retrying in milliseconds.

TODO: accept a function as value, which returns a number.

Scheduler Options

Whenever you invoke the kickq.process() method Kickq will launch the Scheduler. The scheduler is only launched once on the first kickq.process() invocation. The Scheduler's job is to poll the redis server for jobs whose scheduled processing time is nearing the present time.

The scheduler takes care of randomizing the poll interval so that multiple workers will not get easily synchronized whe n polling. You can choose to disable the scheduler or configure it using the folllowing options:

Option :: schedulerOn

Type: boolean Default: true

If this worker will also launch the scheduler loop.

Option :: schedulerInterval

Type: number Default: 1000 milliseconds

The interval the scheduler will check for jobs that need to be queued.

Option :: schedulerFuzz

Type: number Default: 300 milliseconds

The fuzz factor is to create some randomness in the interval so multiple workers will have a chance to spread out evenly. An interval of 1000 with a fuzzy factor of 300 means that the actual interval for scheduler checks will be anywhere between 700 and 1,300 milliseconds.

Option :: schedulerLookAhead

Type: number Default: 1500 milliseconds

When actually performing a db query, how far ahead to look for jobs that need to be moved to the process queue. Rule of thumb = (schedulerInterval + schedulerFuzz) * ~15%.

Logger Options

Kickq exposes a robust logging facility. The messages that kickq generates range from Severe errors, failing database, failed saving etc. down to code debugging messages. The levels are represented by an integer, ranging from 0 (less important) to 1000 (severe).

All levels are exposed as an enum by kickq.LogLevel:

  // Values of the LogLevel Enum
  kickq.LogLevel.SEVERE   = 1000;
  kickq.LogLevel.DATABASE =  900;
  kickq.LogLevel.WARN     =  800;
  kickq.LogLevel.INFO     =  600;
  kickq.LogLevel.FINE     =  400;
  kickq.LogLevel.FINER    =  200;
  kickq.LogLevel.FINEST   =  100;

Read more about the Kickq Logger Facility. Logging can be configured with the following options:

Option :: loggerConsole

Type: boolean Default: false

Log to console too, uses the loggerLevel.

Option :: loggerLevel

Type: boolean Default: 800 (kickq.LogLevel.WARN)

The minimum Logging Level messages that kickq will emit. The debug option overwrites this to 0.

kickq.config('loggerLevel', kickq.LogLevel.INFO);

Option :: loggerFile

Type: boolean Default: false

Save the log to file. By default warning or severe level messages are saved, use the loggerFileLevel to set the verbosity.

Option :: loggerFilename

Type: string Default: "./log/kickq.log"

The logging file name.

Advanced Options

Option :: fetchTimeout

Type: number Default: 0 seconds, 0 for ever

The redis' blpop timeout. Used for fetching the next job in the queue, 0 means indefinite wait.

Option :: guardInterval

Type: number Default: 30000 milliseconds (30s)

How often the Worker Guard to run and check the worker is operating normally.