Home
rethinkdb-job-queue is a persistent job or task queue backed by RethinkDB.
- Find all the details you need about the job queue in this wiki. Menu =>
- See the Change Log for details about what has changed.
- See the documentation WORKLOG for current documentation state.
Queues are an everyday part of life and are more complex than you might think. Queueing Theory has been around since the early 1900s. The 30 page book The Essential Guide to Queueing Theory is a great introduction and well worth the read.
When building an application you will typically need to process many tasks or jobs for one reason or another. Some examples of jobs you may need to process within an application are;
- Sending emails to application users; Registration, News Letters, or confirmation emails for example.
- Mathematical calculations.
- Training machine learning models.
- Image or Video processing.
If the data you need for your jobs is supplied by users of your application, you need to ensure that the jobs can be processed even in the event of an application failure or loss of power to the server. The best example of this is the typical user registration process.
Imagine you have built the next big thing on the internet, however there are not many users yet. If a user decided to register for your application, and you lost their details due to a bug or power outage, that is one user you will possibly never get back.
This is where a job queue comes into the picture. By using a job queue that saves the jobs to persistent storage and manages the state of the job until it is completed, your jobs will be resilient to interruptions.
Another very important feature of a job queue is offloading job processing from your web server. You don't want your web server becoming unresponsive when a user initiates some sort of job in the browser and the web server bogs down whilst working on the job.
Using a distributed processing model you can offload job processing to a host machine designated for processing the jobs as quickly as possible.
With rethinkdb-job-queue the queue jobs are saved to an instance of a RethinkDB database. There are other job queues around that use a Redis database and they are extremely good.
If you decided to build your application using RethinkDB as the NoSQL data store, then you have chosen an extremely flexible, scalable, performant, and realtime database.
By using rethinkdb-job-queue you are removing the need to add other data stores to your application. If you are already using Redis as well for some other data store solution, then your choice of job queue comes down to simply performance and API.
I would urge you to try out the different job queues available on NPM and make an informed decision. Keep in mind the activity of the packages and the number of open issues.
If you spot an issue with this documentation, please fix it so everyone can benefit.