Skip to content

Commit

Permalink
generalized starling client to be a memcachequeue client
Browse files Browse the repository at this point in the history
  • Loading branch information
purzelrakete committed Nov 2, 2008
1 parent cce3786 commit 6a99f1b
Show file tree
Hide file tree
Showing 12 changed files with 47 additions and 34 deletions.
2 changes: 1 addition & 1 deletion CHANGES.markdown
Expand Up @@ -17,7 +17,7 @@ Version 0.2.5, 02.09.08

Version 0.2.4, 08.06.08
- accept both async_ and asynch_ as prefixes for workling method invocation. thank you francois beausoleil!
- added memcached configuration options to starling.yml. see example yml for details. thank you larry diehl!
- added memcached configuration options to workling.yml. see example yml for details. thank you larry diehl!
- re-raise exceptions if there is a problem adding an item to the starling queue. thank you digitalronin!
- added status script for starling client. thank you andrew carter!
- applied patches from dave dupre: http://davedupre.com/2008/03/29/ruby-background-tasks-with-starling-part-2/
Expand Down
6 changes: 3 additions & 3 deletions README.markdown
Expand Up @@ -97,14 +97,14 @@ Workling will now automatically detect and use Starling, unless you have also in

## Starting up the required processes

Here's what you need to get up and started in development mode. Look in config/starling.yml to see what the default ports are for other environments.
Here's what you need to get up and started in development mode. Look in config/workling.yml to see what the default ports are for other environments.

sudo starling -d -p 22122
script/workling_starling_client start

## Configuring starling.yml
## Configuring workling.yml

Workling copies a file called starling.yml into your applications config directory. You can delete this file if you're not planning to use Starling. The config file tells Workling on which port Starling is listening.
Workling copies a file called workling.yml into your applications config directory. You can delete this file if you're not planning to use Starling. The config file tells Workling on which port Starling is listening.

Notice that the default production port is 15151. This means you'll need to start Starling with -p 15151 on production.

Expand Down
17 changes: 14 additions & 3 deletions lib/workling.rb
Expand Up @@ -4,9 +4,17 @@
module Workling
class WorklingError < StandardError; end
class WorklingNotFoundError < WorklingError; end
class StarlingNotFoundError < WorklingError
class QueueserverNotFoundError < WorklingError
def initialize
super "config/starling.yml configured to connect to starling on #{ Workling.config[:listens_on] } for this environment. could not connect to starling on this host:port. pass starling the port with -p flag when starting it. If you don't want to use Starling at all, then explicitly set Workling::Remote.dispatcher (see README for an example)"
super "config/workling.yml configured to connect to queue server on #{ Workling.config[:listens_on] } for this environment. could not connect to queue server on this host:port. for starling users: pass starling the port with -p flag when starting it. If you don't want to use Starling, then explicitly set Workling::Remote.dispatcher (see README for an example)"
end
end

class ConfigurationError < WorklingError
def initialize
super File.exist?(File.join(RAILS_ROOT, 'config', 'starling.yml')) ?
"config/starling.yml has been depracated. rename your config file to config/workling.yml then try again!" :
"config/workling.yml could not be loaded. check out README.markdown to see what this file should contain. "
end
end

Expand Down Expand Up @@ -98,11 +106,14 @@ def self.try_load_a_memcache_client

#
# returns a config hash. reads RAILS_ROOT/config/workling.yml
# NOTE: this used to be starling.yml. simply rename to upgrade.
#
def self.config
config_path = File.join(RAILS_ROOT, 'config', 'workling.yml')
@@config ||= YAML.load_file(config_path)[RAILS_ENV || 'development'].symbolize_keys

# make sure the config file could be read correctly.
raise ConfigurationError.new unless @@config

@@config[:memcache_options].symbolize_keys! if @@config[:memcache_options]
@@config
end
Expand Down
@@ -1,31 +1,33 @@
#
# Wrapper for the starling connection. The connection is made using fiveruns-memcache-client,
# This client can be used for all Queue Servers that speak Memcached, such as Starling.
#
# Wrapper for the memcache connection. The connection is made using fiveruns-memcache-client,
# or memcache-client, if this is not available. See the README for a discussion of the memcache
# clients.
#
# method_missing delegates all messages through to the underlying memcache connection.
#
module Workling
module Clients
class Starling
class MemcacheQueue

# the url with which the memcache client expects to reach starling
attr_accessor :starling_urls
attr_accessor :queueserver_urls

# the memcache connection object
attr_accessor :connection

#
# the client attempts to connect to starling using the configuration options found in
# the client attempts to connect to queueserver using the configuration options found in
#
# Workling.config. this can be configured in config/starling.yml.
# Workling.config. this can be configured in config/workling.yml.
#
# the initialization code will raise an exception if memcache-client cannot connect
# to starling.
# to queueserver.
#
def initialize
@starling_urls = Workling.config[:listens_on].split(',').map { |url| url ? url.strip : url }
options = [@starling_urls, Workling.config[:memcache_options]].compact
@queueserver_urls = Workling.config[:listens_on].split(',').map { |url| url ? url.strip : url }
options = [@queueserver_urls, Workling.config[:memcache_options]].compact
@connection = ::MemCache.new(*options)

raise_unless_connected!
Expand All @@ -37,12 +39,12 @@ def method_missing(method, *args)
end

private
# make sure we can actually connect to starling on the given port
# make sure we can actually connect to queueserver on the given port
def raise_unless_connected!
begin
@connection.stats
rescue
raise Workling::StarlingNotFoundError.new
raise Workling::QeueuserverNotFoundError.new
end
end
end
Expand Down
2 changes: 1 addition & 1 deletion lib/workling/remote/invokers/poller.rb
Expand Up @@ -12,7 +12,7 @@ class Poller
# Seconds to wait while resetting connection
cattr_accessor :reset_time

def initialize(routing, client_class = Workling::Clients::Starling)
def initialize(routing, client_class = Workling::Clients::MemcacheQueue)
Poller.sleep_time = Workling.config[:sleep_time] || 2
Poller.reset_time = Workling.config[:reset_time] || 30

Expand Down
2 changes: 1 addition & 1 deletion lib/workling/remote/runners/starling_runner.rb
Expand Up @@ -20,7 +20,7 @@ class StarlingRunner < Workling::Remote::Runners::Base
cattr_accessor :client

def initialize
StarlingRunner.client = Workling::Clients::Starling.new
StarlingRunner.client = Workling::Clients::MemcacheQueue.new
StarlingRunner.routing = Workling::Routing::ClassAndMethodRouting.new
end

Expand Down
4 changes: 2 additions & 2 deletions lib/workling/return/store/starling_return_store.rb
@@ -1,5 +1,5 @@
require 'workling/return/store/base'
require 'workling/clients/starling'
require 'workling/clients/memcache_queue'

#
# Recommended Return Store if you are using the Starling Runner. This
Expand All @@ -12,7 +12,7 @@ class StarlingReturnStore < Base
cattr_accessor :client

def initialize
self.client = Workling::Clients::Starling.new
self.client = Workling::Clients::MemcacheQueue.new
end

# set a value in the queue 'key'.
Expand Down
2 changes: 1 addition & 1 deletion script/starling_status.rb
Expand Up @@ -11,7 +11,7 @@
trap(:INT) { exit }

begin
client = Workling::Clients::Starling.new
client = Workling::Clients::MemcacheQueue.new
client.reset

client.stats # do this so that connection is shown as established below.
Expand Down
2 changes: 1 addition & 1 deletion test/dispacher_poller_test.rb
Expand Up @@ -22,7 +22,7 @@

client = mock()
client.expects(:get).at_least_once.returns("hi")
Workling::Clients::Starling.expects(:new).at_least_once.returns(client)
Workling::Clients::MemcacheQueue.expects(:new).at_least_once.returns(client)

# Don't take longer than 10 seconds to shut this down.
Timeout::timeout(10) do
Expand Down
@@ -1,23 +1,23 @@
require File.dirname(__FILE__) + '/test_helper'

context "The starling client" do
specify "should be able to connect to multiple Starling instances" do
context "The memcachequeue client" do
specify "should be able to connect to multiple queue instances" do
Workling.send :class_variable_set, "@@config", { :listens_on => "localhost:12345, 127.0.0.1:12346", :memcache_options => { :namespace => "myapp_development" } }
client = Workling::Clients::Starling.new
client = Workling::Clients::MemcacheQueue.new

client.starling_urls.should.equal ["localhost:12345", "127.0.0.1:12346"]
client.queueserver_urls.should.equal ["localhost:12345", "127.0.0.1:12346"]
client.connection.servers.first.host.should == "localhost"
client.connection.servers.first.port.should == 12345
client.connection.servers[1].host.should == "127.0.0.1"
client.connection.servers[1].port.should == 12346
client.connection.namespace.should.equal "myapp_development"
end

specify "should load it's config as well as any given MemCache options from RAILS_ENV/config/starling.yml" do
specify "should load it's config as well as any given MemCache options from RAILS_ENV/config/workling.yml" do
Workling.send :class_variable_set, "@@config", { :listens_on => "localhost:12345", :memcache_options => { :namespace => "myapp_development" } }
client = Workling::Clients::Starling.new
client = Workling::Clients::MemcacheQueue.new

client.starling_urls.should.equal ["localhost:12345"]
client.queueserver_urls.should.equal ["localhost:12345"]
client.connection.servers.first.host.should == "localhost"
client.connection.servers.first.port.should == 12345
client.connection.namespace.should.equal "myapp_development"
Expand All @@ -26,8 +26,8 @@

specify "should load it's config correctly if no memcache options are given" do
Workling.send :class_variable_set, "@@config", { :listens_on => "localhost:12345" }
client = Workling::Clients::Starling.new
client = Workling::Clients::MemcacheQueue.new

client.starling_urls.should.equal ["localhost:12345"]
client.queueserver_urls.should.equal ["localhost:12345"]
end
end
2 changes: 1 addition & 1 deletion test/mocks/client.rb
@@ -1,6 +1,6 @@
module Workling
module Clients
class Starling
class MemcacheQueue
def raise_unless_connected!; end
end
end
Expand Down
2 changes: 1 addition & 1 deletion test/starling_return_store_test.rb
Expand Up @@ -4,7 +4,7 @@
setup do
# the memoryreturnstore behaves exactly like memcache.
MemCache.expects(:new).at_least(0).returns Workling::Return::Store::MemoryReturnStore.new
Workling::Clients::Starling.expects(:connection).at_least(0).returns Workling::Return::Store::MemoryReturnStore.new
Workling::Clients::MemcacheQueue.expects(:connection).at_least(0).returns Workling::Return::Store::MemoryReturnStore.new
end

specify "should be able to store a value with a key, and then retrieve that same value with the same key." do
Expand Down

0 comments on commit 6a99f1b

Please sign in to comment.