Skip to content
Push jobs into starling
Ruby
Find file
Pull request Compare This branch is 35 commits ahead, 2 commits behind joevandyk:master.
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Failed to load latest commit information.
files
lib
tasks
test
MIT-LICENSE
README
Rakefile
init.rb
install.rb

README

= Simplified Starling

This plugin was coded to define an easy way to push jobs into Starling, log 
queues and daemonize the unqueue process.

== Installing Starling

You can install <tt>starling</tt> from the gem server.

    $ gem sources -a http://gems.github.com (you only have to do this once)
    $ sudo gem install starling-starling

== Installing this plugin

Install this Rails plugin as usual.

    script/plugin install git://github.com/fesplugas/simplified_starling.git

After installing the plugin you'll find a configuration file on the config 
folder of your Rails application named +starling.yml+.

    development:
      host: 127.0.0.1
      port: 22122
      ...

Note: If you don't use the `script/plugin install` to install the plugin 
configuration files will not be created. From the root folder of your Rails 
application run the following command to create them.

    $ ruby vendor/plugins/simplified_starling/install.rb

=== How it works?

With +starling+ installed in your machine you can start/stop/restart Starling 
by running.

    $ rake simplified_starling:start
    $ rake simplified_starling:stop
    $ rake simplified_starling:restart

Once Starling is being started you can start processing jobs.

    $ rake simplified_starling:start_processing_jobs

You can start Starling and start processing jobs by ...

    $ rake simplified_starling:start_and_process_jobs

If you want to stop processing jobs ...

    $ rake simplified_starling:stop_processing_jobs

You can put anything into Starling, but after using Starling on a couple of 
projects I've seen I use always the same hash for storing the jobs. This will 
push into the queue the task +test_rendering+ which belongs to a model.

    ##
    # From a model/controller
    #
    newsletter = Newsletter.find(params[:id])
    newsletter.push('test_rendering')
    => STARLING.set('app', { :type => 'Newsletter', 
                             :id => 1, 
                             :task => 'test_rendering' })

You can even push a class task into the queue. This will push into the queue 
the task +recalculate+ which belongs to a model.

    ##
    # From a model/controller
    #
    Stock.push('recalculate')
    => STARLING.set('app', { :type => 'Stock', 
                             :task => 'recalculate' })

To view queue stats ...

    $ rake simplified_starling:stats

Processed jobs are logged.

    $ tail -f log/development_starling.log
    $ tail -f log/production_starling.log

=== Log

Each time a job is pushed and popped to the queue is logged.

    [2008-06-30 11:06:03] Pushed dispatch order
    [2008-06-30 11:06:03] Popped dispatch order

If database connection goes down or dies after a few hours of inactivity 
database connection will be restored and job will be processed.

    [2008-06-30 11:06:42] Pushed rebuild Page 3
    [2008-06-30 11:06:42] WARNING Database connection gone, reconnecting & retrying.
                          {:type=>"Order", :task=>"dispatch", :id=>nil}
    [2008-06-30 11:06:44] Popped rebuild Page 3

If the record you're trying to process is removed from the database before 
the queue is processed you'll see a warning on the logs.

    [2008-06-30 11:06:50] Pushed rebuild Page 3
    [2008-06-30 11:06:50] WARNING Page#3 gone from database.

=== Example 1

Push a +newsletter+ job into +starling+.

    ##
    # app/controllers/typus/newsletters_controller.rb
    #
    class Typus::NewslettersController < TypusController

      def test_deliver
        Newsletter.find(params[:id]).push('test_rendering')
        flash[:notice] = "Newsletter added to queue."
        redirect_to :back
      end

      def deliver
        Newsletter.find(params[:id]).push('deliver')
        flash[:notice] = "Newsletter added to queue."
        redirect_to :back
      end

    end

    ##
    # app/models/newsletter.rb
    #
    class Newsletter < ActiveRecord::Base

      def test_rendering
        ##
        # Your long-running task to deliver newsletter to test rendering
        # in diferent email readers.
      end

      def deliver
        ##
        # Long running task to deliver newsletter ...
        #
      end

    end

=== Example 2

Confirm an +order+ payment and push into +starling+ an stock recalculation job.

    ##
    # app/controllers/payments_controller.rb
    #
    class PaymentsController < ApplicationController

      def confirm
        @order = Order.find_by_token(params[:token])
        @order.confirm_payment
        flash[:notice] = "Thanks for your purchase!"
        redirect_to :action => 'thanks'
      end

    end

    ##
    # app/models/order.rb
    #
    class Order < ActiveRecord::Base

      def confirm_payment
        OrderMailer.deliver_payment_confirmation(order)
        Stock.push('recalculate')
      end

    end

    ##
    # app/models/stock.rb
    #
    class Stock < ActiveRecord::Base

      def self.recalculate
        ##
        # A long running operation ...
        #
      end

    end

=== Example 3

Push a task with options.

    ##
    # app/controllers/repositories_controller.rb
    #
    class RepositoriesController < ApplicationController

      def index
        @repositories = Repository.find(:all, :limit => 100)
      end

      def generate
        100.times do |i|
          token = CGI::Session.generate_unique_id.first(8)
          Repository.push :generate, { :token => token }
        end
        redirect_to :action => 'index'
      end

      def update
        repository = Repository.find_by_token(params[:id])
        token = CGI::Session.generate_unique_id.first(8)
        repository.push :update_token, { :token => token }
        redirect_to :action => 'show', :id => token
      end

    end

    ##
    # app/models/repository.rb
    #
    class Repository < ActiveRecord::Base
    
      def self.generate(options = {})
        # Create repositories with token as it's name ...
        # ...
      end

      def update_token
        # ...
      end

    end

  end

== Acknowledgments

- Blaine Cook, Twitter Inc. for this nice queue system.
- Joe Van Dyk for his work on adding options to tasks, Tanga.com LLC

Copyright (c) 2008 Francesc Esplugas Marti, released under the MIT license
Something went wrong with that request. Please try again.