Skip to content
This repository

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Job queue with MongoDB with emphasis on never losing any task even if worker fails hard (segfault).

branch: master
README.md

Cyclop

Job queue with MongoDB with emphasis on never losing any task even if worker fails hard (segfault).

Dependencies

  • Ruby >= 1.9.2
  • gem "mongo", "~> 1.3.1"
  • gem "posix-spawn", "~> 0.3.6"

Usage

  • Give Cyclop access to mongo:

    Cyclop.db = Mongo::Connection.new["database_name"]
    

    or with Replica-Sets

    Cyclop.db = Mongo::ReplSetConnection.new["database_name"]
    

    or if you're using MongoMapper:

    Cyclop.db = MongoMapper.database
    

    or if you're using Mongoid:

    Cyclop.db = Mongoid.database
    
  • Queue a new task:

    Cyclop.push({
      queue: :upload,
      job_params: {
        url: "http://example.com",
      },
    })
    
  • Queue a new task to process in 5 minutes, to retry 3 times in case of error with a 1 minute delay between each:

    Cyclop.push({
      queue: :convert,
      job_params: {
        tmp_file: "/tmp/uploaded_file_32.png",
      },
      delay: 300,
      retries: 3,
      splay: 60,
    })
    
  • Get next job:

    Cyclop.next
    
  • Get next job on specific queues:

    Cyclop.next :upload, :convert
    
  • Get next job on specific queues for a specific host:

    Cyclop.next :upload, :convert, host: "tartarus.local"
    
  • Get failed jobs (limit to 30):

    Cyclop.failed limit: 30
    
  • Get failed jobs (skip first 10, limit to 30):

    Cyclop.failed skip: 10, limit: 30
    
  • Requeue a failed job:

    job = Cyclop.failed.first
    job.requeue
    
  • Start a worker:

    cyclop -c config.yml
    
  • To get help about the format for config.yml

    cyclop -h
    

Build Status

Build Status

License

cyclop is Copyright © 2011 TalentBox SA. It is free software, and may be redistributed under the terms specified in the LICENSE file.

Something went wrong with that request. Please try again.