public
Description: Push jobs into starling
Clone URL: git://github.com/fesplugas/simplified_starling.git
name age message
file MIT-LICENSE Wed Jun 11 06:27:08 -0700 2008 Initial work ... [fesplugas]
file README Mon Aug 25 07:16:29 -0700 2008 Readme now contains an example of the work done... [fesplugas]
file Rakefile Wed Jun 11 06:27:08 -0700 2008 Initial work ... [fesplugas]
directory files/ Mon Aug 25 05:48:31 -0700 2008 Added queue_pid_file [fesplugas]
file init.rb Sat Jul 12 09:14:56 -0700 2008 Removed line at the end of file [fesplugas]
file install.rb Mon Aug 25 04:59:22 -0700 2008 Use ERB to generate template [fesplugas]
directory lib/ Mon Aug 25 07:00:55 -0700 2008 Async methods now support an arbitrary number o... [fesplugas]
directory tasks/ Mon Aug 25 06:38:22 -0700 2008 If pid file for starling exists, it could be de... [fesplugas]
directory test/ Mon Aug 25 07:00:55 -0700 2008 Async methods now support an arbitrary number o... [fesplugas]
README
= Simplified Starling

This plugin was coded to define an easy way to push jobs into Starling, log 
queues and daemonize the unqueue process.

== Installing Starling

You can install <tt>starling</tt> from the gem server.

    $ gem sources -a http://gems.github.com (you only have to do this once)
    $ sudo gem install starling-starling

== Installing this plugin

Install this Rails plugin as usual.

    script/plugin install git://github.com/fesplugas/simplified_starling.git

After installing the plugin you'll find a configuration file on the config 
folder of your Rails application named +starling.yml+.

    development:
      host: 127.0.0.1
      port: 22122
      ...

Note: If you don't use the `script/plugin install` to install the plugin 
configuration files will not be created. From the root folder of your Rails 
application run the following command to create them.

    $ ruby vendor/plugins/simplified_starling/install.rb

=== How it works?

With +starling+ installed in your machine you can start/stop/restart Starling 
by running.

    $ rake simplified_starling:start
    $ rake simplified_starling:stop
    $ rake simplified_starling:restart

Once Starling is being started you can start processing jobs.

    $ rake simplified_starling:start_processing_jobs

You can start Starling and start processing jobs by ...

    $ rake simplified_starling:start_and_process_jobs

If you want to stop processing jobs ...

    $ rake simplified_starling:stop_processing_jobs

You can put anything into Starling, but after using Starling on a couple of 
projects I've seen I use always the same hash for storing the jobs. This will 
push into the queue the task +test_rendering+ which belongs to a model.

    ##
    # From a model/controller
    #
    newsletter = Newsletter.find(params[:id])
    newsletter.push('test_rendering')
    => STARLING.set('app', { :type => 'Newsletter', 
                             :id => 1, 
                             :task => 'test_rendering' })

You can even push a class task into the queue. This will push into the queue 
the task +recalculate+ which belongs to a model.

    ##
    # From a model/controller
    #
    Stock.push('recalculate')
    => STARLING.set('app', { :type => 'Stock', 
                             :task => 'recalculate' })

To view queue stats ...

    $ rake simplified_starling:stats

Processed jobs are logged.

    $ tail -f log/development_starling.log
    $ tail -f log/production_starling.log

=== Log

Each time a job is pushed and popped to the queue is logged.

    [2008-06-30 11:06:03] Pushed dispatch order
    [2008-06-30 11:06:03] Popped dispatch order

If database connection goes down or dies after a few hours of inactivity 
database connection will be restored and job will be processed.

    [2008-06-30 11:06:42] Pushed rebuild Page 3
    [2008-06-30 11:06:42] WARNING Database connection gone, reconnecting & retrying.
                          {:type=>"Order", :task=>"dispatch", :id=>nil}
    [2008-06-30 11:06:44] Popped rebuild Page 3

If the record you're trying to process is removed from the database before 
the queue is processed you'll see a warning on the logs.

    [2008-06-30 11:06:50] Pushed rebuild Page 3
    [2008-06-30 11:06:50] WARNING Page#3 gone from database.

=== Example 1

Push a +newsletter+ job into +starling+.

    ##
    # app/controllers/typus/newsletters_controller.rb
    #
    class Typus::NewslettersController < TypusController

      def test_deliver
        Newsletter.find(params[:id]).push('test_rendering')
        flash[:notice] = "Newsletter added to queue."
        redirect_to :back
      end

      def deliver
        Newsletter.find(params[:id]).push('deliver')
        flash[:notice] = "Newsletter added to queue."
        redirect_to :back
      end

    end

    ##
    # app/models/newsletter.rb
    #
    class Newsletter < ActiveRecord::Base

      def test_rendering
        ##
        # Your long-running task to deliver newsletter to test rendering
        # in diferent email readers.
      end

      def deliver
        ##
        # Long running task to deliver newsletter ...
        #
      end

    end

=== Example 2

Confirm an +order+ payment and push into +starling+ an stock recalculation job.

    ##
    # app/controllers/payments_controller.rb
    #
    class PaymentsController < ApplicationController

      def confirm
        @order = Order.find_by_token(params[:token])
        @order.confirm_payment
        flash[:notice] = "Thanks for your purchase!"
        redirect_to :action => 'thanks'
      end

    end

    ##
    # app/models/order.rb
    #
    class Order < ActiveRecord::Base

      def confirm_payment
        OrderMailer.deliver_payment_confirmation(order)
        Stock.push('recalculate')
      end

    end

    ##
    # app/models/stock.rb
    #
    class Stock < ActiveRecord::Base

      def self.recalculate
        ##
        # A long running operation ...
        #
      end

    end

=== Example 3

Push a task with options.

    ##
    # app/controllers/repositories_controller.rb
    #
    class RepositoriesController < ApplicationController

      def index
        @repositories = Repository.find(:all, :limit => 100)
      end

      def generate
        100.times do |i|
          token = CGI::Session.generate_unique_id.first(8)
          Repository.push :generate, { :token => token }
        end
        redirect_to :action => 'index'
      end

      def update
        repository = Repository.find_by_token(params[:id])
        token = CGI::Session.generate_unique_id.first(8)
        repository.push :update_token, { :token => token }
        redirect_to :action => 'show', :id => token
      end

    end

    ##
    # app/models/repository.rb
    #
    class Repository < ActiveRecord::Base
    
      def self.generate(options = {})
        # Create repositories with token as it's name ...
        # ...
      end

      def update_token
        # ...
      end

    end

  end

== Acknowledgments

- Blaine Cook, Twitter Inc. for this nice queue system.
- Joe Van Dyk for his work on adding options to tasks, Tanga.com LLC

Copyright (c) 2008 Francesc Esplugas Marti, released under the MIT license