Fast background job processing
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
spec initial release Dec 4, 2018
src initial release Dec 4, 2018
.editorconfig initial release Dec 4, 2018
.gitignore initial release Dec 4, 2018
.travis.yml initial release Dec 4, 2018
LICENSE initial release Dec 4, 2018
README.md docs (README): update example Dec 4, 2018
shard.yml initial release Dec 4, 2018

README.md

Onyx::Background

Built with Crystal Travis CI build API docs Latest release

A fast background job processing for Crystal.

About

This is a Redis-based background jobs processing for Crystal alternative to sidekiq and mosquito. It's a component of Onyx Framework, but can be used separately.

Features

This publicly available open-source version has the following functionality:

  • Enqueuing jobs for immediate processing
  • Enqueuing jobs for delayed processing (i.e. in: 5.minutes or at: Time.now + 1.hour)
  • Different job queues ("default" by default)
  • Concurrent jobs processing (with a separate Redis client for each fiber)
  • Verbose Redis logging of all the activity (i.e. every attempt made)
  • Moving stale jobs (i.e. with dead workers) to the failed list

Performance

Thorough benchmaring is to be done yet, however, currently a single Worker is able to process more than 7500 jobs per second on my 0.9GHz machine with Redis instance running on itself.

Installation

Onyx::Background works with Redis version ~> 5.0.

Add this to your application's shard.yml:

dependencies:
  onyx-background:
    github: onyxframework/background
    version: ~> 0.1.0

This shard follows Semantic Versioning v2.0.0, so check releases and change the version accordingly.

Usage

API docs

Please refer to API documentation available online: https://api.onyxframework.org/background

Note: it is updated on every master branch commit

Example

require "onyx-background"

struct Jobs::Nap
  include Onyx::Background::Job

  def initialize(@sleep : Int32 = 1)
  end

  def perform
    sleep(@sleep)
  end
end

manager = Onyx::Background::Manager.new
manager.enqueue(Jobs::Nap.new)

puts "Enqueued"

logger = Logger.new(STDOUT, Logger::DEBUG)
worker = Onyx::Background::Worker.new(logger: logger)
worker.run
$ crystal app.cr
Enqueued
I -- worker: Working...
D -- worker: Waiting for a new job...
D -- worker: [fa5b6d65-46fe-4c88-829f-d69023c4c6de] Attempting
D -- worker: [fa5b6d65-46fe-4c88-829f-d69023c4c6de] Performing Jobs::Nap {"sleep":1}...
D -- worker: Waiting for a new job...
D -- worker: [fa5b6d65-46fe-4c88-829f-d69023c4c6de] Completed

It's also highly recommended to run a Watcher process to watch for stale jobs. If you're queuing delayed jobs, the Watcher is required to move the jobs to the ready queue on time:

# watcher.cr
require "onyx-background"

watcher = Onyx::Background::Watcher.new
watcher.run

Development

Redis is flushed during the spec, so you must specify a safe-to-flush Redis database in the REDIS_URL. To run the specs, use the following command:

$ env REDIS_URL=redis://localhost:6379/1 crystal spec

Contributing

  1. Fork it (https://github.com/onyxframework/background/fork)
  2. Create your feature branch (git checkout -b my-new-feature)
  3. Commit your changes (git commit -am 'Add some feature')
  4. Push to the branch (git push origin my-new-feature)
  5. Create a new Pull Request

Contributors

Licensing

This software is licensed under BSD 3-Clause License with "Commons Clause" License Condition v1.0. See LICENSE.