Job queue with MongoDB with emphasis on never losing any task even if worker fails hard (segfault).
Ruby
Permalink
Failed to load latest commit information.
bin Add die_after option to worker (limit number of job processed before … Jun 28, 2011
lib Specify license in gemspec and bump version to 0.1.8 Jul 16, 2013
spec Fix spec Aug 11, 2011
.gitignore Add Gemfile.lock in repository Sep 23, 2011
.rspec Add LICENSE, remove rubyforge_project Jun 10, 2011
.travis.yml Remove JRuby from Travis Aug 11, 2011
Gemfile
Gemfile.lock Updated Gemfile.lock Oct 16, 2011
LICENSE Specify license in gemspec and bump version to 0.1.8 Jul 16, 2013
README.md
Rakefile Add meta redirect for doc May 30, 2011
cyclop.gemspec Specify license in gemspec and bump version to 0.1.8 Jul 16, 2013

README.md

Cyclop

Job queue with MongoDB with emphasis on never losing any task even if worker fails hard (segfault).

Dependencies

  • Ruby >= 1.9.2
  • gem "mongo", "~> 1.3.1"
  • gem "posix-spawn", "~> 0.3.6"

Usage

  • Give Cyclop access to mongo:

    Cyclop.db = Mongo::Connection.new["database_name"]
    

    or with Replica-Sets

    Cyclop.db = Mongo::ReplSetConnection.new["database_name"]
    

    or if you're using MongoMapper:

    Cyclop.db = MongoMapper.database
    

    or if you're using Mongoid:

    Cyclop.db = Mongoid.database
    
  • Queue a new task:

    Cyclop.push({
      queue: :upload,
      job_params: {
        url: "http://example.com",
      },
    })
    
  • Queue a new task to process in 5 minutes, to retry 3 times in case of error with a 1 minute delay between each:

    Cyclop.push({
      queue: :convert,
      job_params: {
        tmp_file: "/tmp/uploaded_file_32.png",
      },
      delay: 300,
      retries: 3,
      splay: 60,
    })
    
  • Get next job:

    Cyclop.next
    
  • Get next job on specific queues:

    Cyclop.next :upload, :convert
    
  • Get next job on specific queues for a specific host:

    Cyclop.next :upload, :convert, host: "tartarus.local"
    
  • Get failed jobs (limit to 30):

    Cyclop.failed limit: 30
    
  • Get failed jobs (skip first 10, limit to 30):

    Cyclop.failed skip: 10, limit: 30
    
  • Requeue a failed job:

    job = Cyclop.failed.first
    job.requeue
    
  • Start a worker:

    cyclop -c config.yml
    
  • To get help about the format for config.yml

    cyclop -h
    

Build Status

Build Status

License

cyclop is Copyright © 2011 TalentBox SA. It is free software, and may be redistributed under the terms specified in the LICENSE file.