poor performance of 1.7.4 compared to 1.5.0

[alpinegizmo]

We are seeing poor performance with recent versions of memcache-client, including 1.7.4. We have been trying to figure out what's going on since upgrading from Rails 2.2 to 2.3 a few weeks ago. In the context of our Rails application, memcache is now approximately 5x slower than it was with version 1.5.x (with timeout nil in both cases).

This simplistic benchmark also shows a performance degradation -- about 2x as slow -- when comparing versions 1.5.0 and 1.7.4 of the memcache-client gem:

require 'rubygems'
require 'benchmark'
require 'memcache'

class Array
  def sum
    inject(0) {|sum, x| sum += x; sum}
  end
end

m = MemCache.new(['localhost:11211'], {:namespace => 'dude'})
m.set('key', 'valuevalue')
times = []
10000.times do
  times << Benchmark.measure {m.get 'key'}.real
end
puts "min: #{times.min} max: #{times.max} mean: #{times.sum / times.size}"

[alpinegizmo]

I have found that if I use this simple benchmark to measure memcache 1.7.4 performance inside of (1) irb, or (2) a rails console inside a new, empty app, or (3) inside our large app, memcache gets progressively slower (the empty rails app is about 2x slower than naked irb, and our large app is 4-5x slower). I suspect the problem is garbage collection. My theory is that recent versions of memcache-client are creating more garbage, and the impact is felt more in larger applications because the heap is larger and GC takes longer.

[mperham]

If you have a large app, you should have multiple memcached servers. Try benchmarking 1.5.0 with multiple servers. You'll find a new definition of slow.

Aside from that comment, there's nothing really I can do with this issue. 1.5.0 is horrible when it comes to fault tolerance and outages. Reliability is more important than speed in most cases. You can always go to fauna/memcached if you need absolute speed.

[alpinegizmo]

We do have 4 memcache servers, which we were using with 1.5.x, then 1.7.4, and now with fauna/memcached. Switching to Evan Weaver's library has got us back to the same performance we enjoyed with 1.5.x (or slightly better). We monitor our app's performance pretty carefully, and what I can tell you is that 1.7.4 had a very negative impact on our application because we ended up spending a lot more time doing garbage collection. Any given memcache fetch wasn't particularly slower, but the overall system impact was huge -- costing us 20-30msec per page request on average. Perhaps switching from C to Ruby cost more than you realize. The impact of garbage collection seems small in the context of simple benchmarks, but in an app with a large heap, each GC takes a lot longer.

[mperham]

If you are using 4 servers (e.g. MemCache.new %w(mc1 mc2 mc3 mc4)) then there's no way 1.5.0 is faster than 1.7.4. Add '127.0.0.1:11211' to the array of servers in your benchmark and you'll find that 1.5.0 slows down dramatically. Here's my results:

(1.5.0) min: 0.000190973281860352 max: 0.0202651023864746 mean: 0.000254420280456543
(1.7.4) min: 0.000129938125610352 max: 0.0172569751739502 mean: 0.000226797008514404

Yes, in the simplest case (a single server with no failover or error handling), 1.5.0 is faster than 1.7.4. It sounds like you have the desire and knowledge to fix this - I would happily accept patches if you profile it and find an issue.