Skip to content
This repository

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

A benchmark between a JRuby Hash, Hazelcast and Infinispan.

branch: master
README.textile

Hash vs Hazelcast vs Infinispan

Overview

This is just a simple benchmark to compare Hazelcast and Infinispan performance (with defaults), and includes a regular Hash object as a baseline.

Hazelcast

A persistent (to disk) Key/Value store with strong availability guarantees. Turns out, it’s not very fast relative to other popular Key/Value stores like Redis. At about 3.5ms for a key fetch over 10,000 keys, it’s not exactly slow as that’s moderately faster than you might expect for an RDBMS, but it’s performance makes it the slowest K/V store I’ve tested so far.

If you value Durability, Consistency, and High Availability in an Embedded K/V store while still maintaining better-than-RDBMS performance, then Hazelcast might be what you’re looking for. It’s worth mentioning that Hazelcast should also scale horizontally very well.

Hazelcast only requires a single JAR, which makes it very easy to get started with.

Infinispan

Intended to be a straight up cache, compatible with the MemCache API. It’s data set is in memory, supports locality (Hazelcast’s NearCache) and a more advanced eviction algorithm (LIRS; no idea how much this contributes to overall performance really). For the embedded instance in this benchmark it’s only about half the speed of a JRuby Hash, which is very impressive.

Infinispan should also scale out horizontally very well as Maps can be “sharded”, nodes can be rack aware (through configuration) and keys locally cached on access.

Infinispan requires nine JARs to run an embedded instance, which is kind of the opposite of “impressive”. Maven makes that a non-issue if you’re deploying as a unified JAR, but if you intend to run as a more traditional file-based Ruby project you have to decide wether you feel like copying dependencies at deploy, or committing all those JARs to your repository.

Project Structure

As a JRuby example this demonstrates a few things:

  • We have dependency entries for Hazelcast and Infinispan in our pom.xml
  • The jruby-complete JAR is bundled in, so all you need is Java (and Maven)
  • src/main/ruby/jar-bootstrap.rb is present, so JRuby will automatically use it as the entry-point
  • When you package everything up into a unified JAR, you don’t need to require separate JARs (it’s kind of like unpacking a GEM into your project)
  • java_import will bring those classes into scope for convenient usage
  • Actually using these two different projects is trivial; once you’ve instantiated a Map/Hash, the interface is the same as a regular Ruby Hash

Notes on Serialization

Infinispan seems to handle serializing “primitives”, Arrays and Hashes just fine. Hazelcast throws errors serializing Ruby Hashes. They have to be converted to Java HashMaps by wrapping them like so:

cache['foo'] = java.util.HashMap.new({ 'bar' => 'baz' })

Hazelcast also seems to have trouble serializing Ruby Arrays, at least when used as keys, so they have to be converted to Java Arrays first:

cache[ [1293624000000, "67bb5d5f2fddbea2bf24a29a5f00aa56"].to_java ] = "Some document..."

Other than that, they seem about the same. I haven’t tested either with serialized Ruby objects, but that’s not my use-case anyway.

Usage

From the root of the project run:

mvn clean install

Then you can just start it up:

java -jar target/map-benchmarks-1.0.0.jar

Results

On a Core 2 Duo laptop running at 2.66GHz these are my results:

Rehearsal --------------------------------------------------
Hash:set         0.280000   0.010000   0.290000 (  0.223000)
Hazelcast:set   34.890000   2.200000  37.090000 ( 36.979000)
Infinispan:set   0.900000   0.030000   0.930000 (  0.628000)
Hash:get         0.280000   0.010000   0.290000 (  0.176000)
Hazelcast:get   31.400000   1.280000  32.680000 ( 32.997000)
Infinispan:get   0.230000   0.010000   0.240000 (  0.147000)
---------------------------------------- total: 71.520000sec

                     user     system      total        real
Hash:set         0.010000   0.000000   0.010000 (  0.011000)
Hazelcast:set   60.140000   2.550000  62.690000 ( 63.818000)
Infinispan:set   0.110000   0.010000   0.120000 (  0.098000)
Hash:get         0.010000   0.000000   0.010000 (  0.007000)
Hazelcast:get   30.130000   1.180000  31.310000 ( 31.637000)
Infinispan:get   0.020000   0.000000   0.020000 (  0.018000)
Something went wrong with that request. Please try again.