Permalink
Browse files

Initial work on network edgecase testing

  • Loading branch information...
1 parent 2f2411e commit 59cc8717f963108b173dea08a044500094e47c1b @mperham mperham committed Aug 20, 2010
Showing with 66 additions and 53 deletions.
  1. +1 −0 .rvmrc
  2. +24 −0 Performance.md
  3. +10 −4 README.md
  4. +1 −0 dalli.gemspec
  5. +9 −48 lib/dalli/ring.rb
  6. +5 −1 lib/dalli/server.rb
  7. +16 −0 test/test_network.rb
View
@@ -0,0 +1 @@
+rvm use 1.9.2@dalli --create
View
@@ -0,0 +1,24 @@
+Performance
+====================
+
+Caching is all about performance, so I carefully track Dalli performance to ensure no regressions.
+
+ Testing 1.8.5 with ruby 1.9.2p0 (2010-08-18 revision 29036) [x86_64-darwin10.4.0]
+ user system total real
+ set:plain:memcache-client 1.600000 0.390000 1.990000 ( 2.020491)
+ set:ruby:memcache-client 1.680000 0.390000 2.070000 ( 2.108217)
+ get:plain:memcache-client 1.740000 0.250000 1.990000 ( 2.018315)
+ get:ruby:memcache-client 1.790000 0.250000 2.040000 ( 2.065529)
+ multiget:ruby:memcache-client 0.800000 0.090000 0.890000 ( 0.914336)
+ missing:ruby:memcache-client 1.480000 0.250000 1.730000 ( 1.761555)
+ mixed:ruby:memcache-client 3.470000 0.640000 4.110000 ( 4.195236)
+
+ Testing 0.1.0 with ruby 1.9.2p0 (2010-08-18 revision 29036) [x86_64-darwin10.4.0]
+ user system total real
+ set:plain:dalli 0.430000 0.180000 0.610000 ( 1.051395)
+ set:ruby:dalli 0.490000 0.180000 0.670000 ( 1.124848)
+ get:plain:dalli 0.490000 0.210000 0.700000 ( 1.141887)
+ get:ruby:dalli 0.540000 0.200000 0.740000 ( 1.188353)
+ multiget:ruby:dalli 0.510000 0.200000 0.710000 ( 0.772860)
+ missing:ruby:dalli 0.450000 0.210000 0.660000 ( 1.070748)
+ mixed:ruby:dalli 1.050000 0.390000 1.440000 ( 2.304933)
View
@@ -5,6 +5,7 @@ Dalli is a high performance pure Ruby client for accessing memcached servers. I
The name is a variant of Salvador Dali for his famous painting [The Persistence of Memory](http://en.wikipedia.org/wiki/The_Persistence_of_Memory).
+
Design
------------
@@ -22,6 +23,7 @@ So a few notes. Dalli:
2. contains explicit "chokepoint" methods which handle all requests; these can be hooked into by monitoring tools (NewRelic, Rack::Bug, etc) to track memcached usage.
3. comes with hooks to replace memcache-client in Rails.
+
Installation and Usage
------------------------
@@ -32,24 +34,27 @@ Installation and Usage
dc.set('abc', 123)
value = dc.get('abc')
+
Usage with Rails
---------------------------
In your Gemfile:
gem 'dalli'
-In `config/environments/production.rb`:
+In `config/environments/production.rb`. Note that we are also setting a reasonable default for maximum cache entry lifetime (one day), enabling compression for large values, and namespacing all entries for this rails app. Remove the namespace if you have multiple apps which share cached values.
- config.cache_store = :dalli_store, 'localhost:11211'
+ require 'active_support/cache/dalli_store'
+ config.cache_store = :dalli_store, 'cache-1.example.com', 'cache-2.example.com',
+ :namespace => NAME_OF_RAILS_APP, :expires_in => 1.day, :compress => true, :compress_threshold => 64.kilobytes
Features and Changes
------------------------
memcache-client allowed developers to store either raw or marshalled values with each API call. I feel this is needless complexity; Dalli allows you to control marshalling per-Client with the `:marshal => false` flag but you cannot explicitly set the raw flag for each API call. By default, marshalling is enabled.
-ActiveSupport::Cache implements several esoteric features so there is no need for Dalli to reinvent them. Specifically, key namespaces and automatic pruning of keys longer than 250 characters.
+I've removed support for key namespaces and automatic pruning of keys longer than 250 characters. ActiveSupport::Cache implements these features so there is little need for Dalli to reinvent them.
By default, Dalli is thread-safe. Disable thread-safety at your own peril.
@@ -59,7 +64,8 @@ Note that Dalli does not require ActiveSupport or Rails. You can safely use it
Author
----------
-Mike Perham, mperham@gmail.com, [mikeperham.com](http://mikeperham.com), [@mperham](http://twitter.com/mperham)
+Mike Perham, mperham@gmail.com, [mikeperham.com](http://mikeperham.com), [@mperham](http://twitter.com/mperham) If you like and use this project, please
+give me a recommendation at [WWR](http://workingwithrails.com/person/10797-mike-perham). Happy caching!
Copyright
View
@@ -15,6 +15,7 @@ Gem::Specification.new do |s|
"README.md",
"Rakefile",
"Gemfile",
+ "Performance.md",
]
s.homepage = %q{http://github.com/mperham/dalli}
s.rdoc_options = ["--charset=UTF-8"]
View
@@ -60,20 +60,6 @@ def hash_for(key)
Zlib.crc32(key)
end
- class Entry
- attr_reader :value
- attr_reader :server
-
- def initialize(val, srv)
- @value = val
- @server = srv
- end
-
- def inspect
- "<#{value}, #{server.host}:#{server.port}>"
- end
- end
-
def entry_count_for(server, total_servers, total_weight)
((total_servers * POINTS_PER_SERVER * server.weight) / Float(total_weight)).floor
end
@@ -98,41 +84,16 @@ def self.binary_search(ary, value)
end
return upper
end
-
- # Native extension to perform the binary search within the ring. This is purely optional for
- # performance and only necessary if you are using multiple memcached servers.
- class << self
- begin
- require 'inline'
- inline do |builder|
- builder.c <<-EOM
- int binary_search(VALUE ary, unsigned int r) {
- int upper = RARRAY_LEN(ary) - 1;
- int lower = 0;
- int idx = 0;
- ID value = rb_intern("value");
-
- while (lower <= upper) {
- idx = (lower + upper) / 2;
-
- VALUE continuumValue = rb_funcall(RARRAY_PTR(ary)[idx], value, 0);
- unsigned int l = NUM2UINT(continuumValue);
- if (l == r) {
- return idx;
- }
- else if (l > r) {
- upper = idx - 1;
- }
- else {
- lower = idx + 1;
- }
- }
- return upper;
- }
- EOM
- end
- rescue Exception => e
+
+ class Entry
+ attr_reader :value
+ attr_reader :server
+
+ def initialize(val, srv)
+ @value = val
+ @server = srv
end
end
+
end
end
View
@@ -41,7 +41,7 @@ def unlock!
def down!
close
- @down_at = Time.now
+ @down_at = Time.now.to_i
@msg = $!.message
nil
end
@@ -52,6 +52,9 @@ def down!
def connection
@sock ||= begin
+ if @down_at && @down_at == Time.now.to_i
+ raise Dalli::NetworkError, "#{self.hostname}:#{self.port} is currently down: #{@msg}"
+ end
begin
s = TCPSocket.new(hostname, port)
s.setsockopt Socket::IPPROTO_TCP, Socket::TCP_NODELAY, 1
@@ -63,6 +66,7 @@ def connection
s
rescue SocketError, SystemCallError, IOError, Timeout::Error
down!
+ raise Dalli::NetworkError, "#{self.hostname}:#{self.port} is currently down: #{@msg}"
end
end
end
View
@@ -0,0 +1,16 @@
+require 'helper'
+
+class TestNetwork < Test::Unit::TestCase
+ context 'assuming a bad network' do
+ setup do
+ end
+
+ should 'handle connection refused' do
+ assert_raises Dalli::NetworkError do
+ dc = Dalli::Client.new 'localhost:19122'
+ dc.get 'foo'
+ end
+ end
+
+ end
+end

0 comments on commit 59cc871

Please sign in to comment.