An AMQP-based background worker system for Ruby.
Ruby
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Failed to load latest commit information.
doc/example
lib
spec
.gitignore
.travis.yml
Gemfile
Guardfile
MIT-LICENSE
PROGRESS-NOTES.txt
README.markdown
Rakefile
THOUGHTS
woodhouse.gemspec

README.markdown

Woodhouse

Build Status

A RabbitMQ-based background worker system for Ruby designed to make managing heterogenous tasks relatively easy.

The use case for Woodhouse is for reliable and sane performance in situations where jobs on a single queue may vary significantly in length. The goal is to permit large numbers of quick jobs to be serviced even when many slow jobs are in the queue. A secondary goal is to provide a sane way for jobs on a given queue to be given special priority or dispatched to a server more suited to them.

Woodhouse 1.0, located in the 1-0-stable branch, is production-ready and stable for Ruby 1.9. The master branch includes development on Woodhouse 2.0, which targets Ruby 2.0 or later.

Please look at the wiki for documentation.

Features

  • Configurable worker sets per server
  • Configurable number of threads per worker
  • Segmenting a single queue among multiple workers based on job characteristics (using AMQP header exchanges)
  • Extension system
  • Progress reporting on jobs with the progress extension
  • New Relic background job reporting with the new_relic extension
  • Live status reporting with the status extension
  • Job dispatch and execution middleware stacks

Upcoming

  • Live reconfiguration of workers -- add or remove workers across one or more nodes without restarting
  • Persistent configuration changes -- configuration changes saved to a data store and kept across deploys
  • Web interface

Acknowledgements

Woodhouse originated in a substantially modified version of the Workling background worker system, although all code has since been replaced.

This library was developed for CrowdCompass and was released as open source with their permission.