Kick Jobs out the door, quickly!
A Robust Node.js queueing service based on redis.
npm install kickq --save
The full documentation can be found in the project's wiki.
The Core API
Utilities and Helpers
Notes on v1.x.x Upgrade
On August 23rd 2015 Kickq got a major version bump from
v1.0.0. The whole of the Kickq codebase has been rehauled and updated using the latest libraries and patterns in Node.js, no API changes were made, Kickq should continue to operate as it was. A very small change has happened in a single processing object attribute which is insignificant and you probably didn't ever care about, id you did read more about it in the wiki.
Moved to Bluebird Promises library
We've let go of when in favor of Bluebird. This is a major change in how Kickq operates as when Kickq was first built Promise libraries where not very performant so we used a specific version of When (1.8.1) which resolved promises Synchronously (vs asynchronously as the spec dictates). We've come a long way since then so the core Promise library change was long overdue.
More robust Promises patterns were applied throughout the codebase.
Refactored Worker Controller
The Worker controller is the module responsible for fetching job from the queue and providing them to the consumer worker for processing. The whole system has been refactored and rehauled to be more robust and handle all outcomes (error or not) more diligently and specifically.
Redis connectivity monitor
We've created a master Redis Connectivity monitor which is responsible for letting the rest of the system know of Redis Connectivity State and emit events when a change occurs, Redis goes down or comes up.
Thus Kickq will now handle Redis disconnects way more softly and patiently wait for the connection to be re-established.
We've lowered the logging verbosity quite a bit and restructured the logs so they make more sense.
The whole of the codebase has been brought up to 2015 Node state in regards to styling.
- v1.1.0, 05 Feb 2016
- Removed node-syslog package and logging to syslog functionality.
- Upgraded all packages to latest.
- v1.0.0, 23 Aug 2015
- Major rehaul and refactoring of Kickq.
- Changed core Promises Library from When to Bluebird.
- Rewritten the Worker Controller which is responsible for fetching and providing the jobs to the consumer worker.
- Introduced the master Redis Connectivity monitor, Kickq now handles disconnects way more gracefully, and will restore operations when Redis re-connection occures.
- Reduced Logging verbosity and tweaked existing one to be more informant.
- v0.2.4, 30 Jun 2015
- Lightened up the npm install bundle using npmingore and better devdeps, thankyou @lbeschastny
- v0.2.3, 16 Jun 2015
- Accept any truthy value as an error callback in job processing.
- v0.2.2, 22 Apr 2015
- Restore node-syslog package and test vs 0.12 in travis.
- v0.2.1, 04 Apr 2015
- Set the log level properly.
- v0.2.0, 26 Mar 2015
- Made Kickq compatible with node 0.12.0 and incompatible with node 0.8.0, thankyou @lbeschastny
- v0.1.0, 17 Feb 2015
- Updated all dependencies to latest.
- Adjusted unit tests.
- Poke poke, is this on?
- v0.0.16, 03 May 2013
- Fixed back in
.get()method, not invoking callback when no results.
kickq.Errorfor semantic correctness.
- Cleaned up error objects.
- Fixed back in
- v0.0.15, 27 Apr 2013
- Now publishes on
kickq:deletefor all job ites that get deleted.
- Delete now removes all traces of job item.
- Bug fix when job failed and test case to avoid regression.
- Now publishes on
- v0.0.14, 27 Apr 2013
- Added time index.
- Fixed state bug when job failed.
- More debugging logs.
- v0.0.13, 26 Apr 2013
- Global Job Item configuration options were not applied to new job items, fixed.
- Bug fixes on Job Item when processing timedout.
- More test cases.
- v0.0.12, 26 Apr 2013
- Perform state record creation.
- More logging for debugging purposes.
- v0.0.10, 26 Apr 2013
- Simplified logger level configuration.
- More logging messages.
- Lax stress tests margins.
- v0.0.9, 26 Apr 2013
- A mythical bug of epic proportions, redis client connected before config was set.
- v0.0.7, 24 Apr 2013
- Enabled purging of completed jobs.
- More tests for job item.
- More logging is performed when kickq fails.
- v0.0.6, 24 Apr 2013
- Fixed metrics stop bug.
- Exposed more internal libs.
- Auto-fetches ver info.
- v0.0.5, 23 Apr 2013
- Created the Metrics module.
- Several bug fixes.
- Exposed more internal parts for better plugin integration.
- v0.0.4, 18 Apr 2013
- Created Worker Guard that ensures worker's health.
- kickq is now resilient to redis disconnections and errors.
- Worker error throttler refactored.
- Now saves to syslog instead of db.
- v0.0.2, 16 Apr 2013
- Added Logging facility.
- Performance optimizations.
- Implemented the
.get()method for fetching job items.
- v0.0.1, 11 Apr 2013
- Big Bang
Copyright 2016 Verbling (Fluency Forums Corporation)
Licensed under the MIT License