Skip to content
This repository

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

A Rails plugin/gem to run asynchronous processes in a configurable pipeline

branch: master

Fetching latest commit…

Octocat-spinner-32-eaf2f5

Cannot retrieve the latest commit at this time

Octocat-spinner-32 examples
Octocat-spinner-32 generators
Octocat-spinner-32 lib
Octocat-spinner-32 spec
Octocat-spinner-32 .gitignore
Octocat-spinner-32 CHANGELOG
Octocat-spinner-32 LICENSE
Octocat-spinner-32 README.rdoc
Octocat-spinner-32 Rakefile
Octocat-spinner-32 TODO
Octocat-spinner-32 VERSION
Octocat-spinner-32 init.rb
Octocat-spinner-32 pipeline.gemspec
README.rdoc

Pipeline

Description

Pipeline is a Rails plugin/gem to run asynchronous processes in a configurable pipeline.

Documentation

rdoc.info/projects/dtsato/pipeline

Features

  • Execution of sequential user-defined stages in an asynchronous pipeline

  • Persistence of pipeline instances and stages

  • Error recovery strategies:

    • Irrecoverable errors fail the entire pipeline

    • Recoverable errors are automatically retried (using dj's exponential retry strategy)

    • Recoverable errors that require user input pause the pipeline for further retry

  • Cancelling/Resuming of a paused pipeline

  • Callbacks before and after executing stages and pipeline

Installation and Use

pipeline can be installed as either a RubyGem (recommended) or as a plugin.

Gem

To install pipeline as a RubyGem, add the following lines to your config/environment.rb file:

config.gem "pipeline"
config.gem "delayed_job"

And execute:

rake gems:install
rake gems:unpack # Optional, if you want to vendor the gem

Plugin

To install it as a plugin, run:

script/plugin install git://github.com/dtsato/pipeline.git

Generating the required tables

In order to persist your pipelines and stages, execute the following:

script/generate pipeline # To generate the migration scripts that will store pipelines
script/generate delayed_job # To generate the migration scripts for delayed_job
rake db:migrate

Starting your workers

You will also need to run your Delayed Job workers that will process the pipeline jobs in the background:

rake jobs:work

Dependencies

Usage

Check examples for more examples (including error-recovery and cancelling)

class Step1 < Pipeline::Stage::Base
  def perform
    puts("Started step 1")
    sleep 2
    puts("Finished step 1")
  end
end

class Step2 < Pipeline::Stage::Base
  def perform
    puts("Started step 2")
    sleep 3
    puts("Finished step 2")
  end
end

class TwoStepPipeline < Pipeline::Base
  define_stages Step1 >> Step2
end

Pipeline.start(TwoStepPipeline.new)

Copyright

Copyright © 2009 Danilo Sato. See LICENSE for details.

Something went wrong with that request. Please try again.