Benchmarks

Ben Manes edited this page Nov 9, 2015 · 30 revisions

The benchmarks use Java microbenchmark harness to provide an accurate analysis. The caches are configured as,

  • Caffeine and ConcurrentLinkedHashMap size their internal structures based on the number of CPUs
  • Guava was configured with a concurrency level of 64 (defaults to 4 to reduces memory usage). Note that Guava #2063 would resolve the poor performance, but has been backlogged for years (up to 25x faster!).
  • Ehcache v2 is internally hardcoded to 100 segments, whereas v3 is not segmented
  • Infinispan "old" is a Guava-like cache configured with a concurrency level of 64
  • Infinispan "new" is a rewrite using a lock-free deque (default in v7.2+)

Desktop-class

Run on a MacBook Pro i7-4870HQ CPU @ 2.50GHz (4 core) 16 GB Yosemite.

Compute

In this benchmark the caches are unbounded, fully populated, and the computation returns a constant value. This benchmark demonstrates the overhead due to locking when the entry is present. Caffeine performs a lock-free prescreening before falling back to an atomic compute if absent call. The scenarios graphed are all threads retrieving a single entry ("sameKey") and threads retrieving different keys based on a Zipf distribution ("spread").

Read (100%)

In this benchmark 8 threads concurrently read from a cache configured with a maximum size.

Read (75%) / Write (25%)

In this benchmark 6 threads concurrently read from and 2 threads write to a cache configured with a maximum size.

Write (100%)

In this benchmark 8 threads concurrently write to a cache configured with a maximum size.

Server-class

The benchmarks were run on an Azure G4 instance, the largest available during a free trial from the major cloud providers. The machine was reported as a single socket Xeon E5-2698B v3 @ 2.00GHz (16 core, hyperthreading disabled), 224 GB, Ubuntu 15.04.

Compute

Cache same key spread
ConcurrentHashMap 29,679,839 65,726,864
Caffeine 1,581,524,763 530,182,873
Guava 25,132,366 114,608,951

Read (100%)

Unbounded ops/s (8 threads) ops/s (16 threads)
ConcurrentHashMap (v8) 560,367,163 1,171,389,095
ConcurrentHashMap (v7) 301,331,240 542,304,172
Bounded
Caffeine 181,703,298 382,355,194
ConcurrentLinkedHashMap 154,771,582 313,892,223
LinkedHashMap_Lru 9,209,065 13,598,576
Guava (default) 12,434,655 10,647,238
Guava (64) 24,533,922 43,101,468
Ehcache2_Lru 11,252,172 20,750,543
Ehcache3_Lru 11,415,248 17,611,169
Infinispan_Old_Lru 29,073,439 49,719,833
Infinispan_New_Lru 4,888,027 4,749,506

Read (75%) / Write (25%)

Unbounded ops/s (8 threads) ops/s (16 threads)
ConcurrentHashMap (v8) 441,965,711 790,602,730
ConcurrentHashMap (v7) 196,215,481 346,479,582
Bounded
Caffeine 144,193,725 279,440,749
ConcurrentLinkedHashMap 63,968,369 122,342,605
LinkedHashMap_Lru 8,668,785 12,779,625
Guava (default) 11,782,063 11,886,673
Guava (64) 22,782,431 37,332,090
Ehcache2_Lru 9,472,810 8,471,016
Ehcache3_Lru 10,958,697 17,302,523
Infinispan_Old_Lru 22,663,359 37,270,102
Infinispan_New_Lru 4,753,313 4,885,061

Write (100%)

Unbounded ops/s (8 threads) ops/s (16 threads)
ConcurrentHashMap (v8) 60,477,550 50,591,346
ConcurrentHashMap (v7) 46,204,091 36,659,485
Bounded
Caffeine 55,281,751 48,295,360
ConcurrentLinkedHashMap 23,819,597 39,797,969
LinkedHashMap_Lru 10,179,891 10,859,549
Guava (default) 4,764,056 5,446,282
Guava (64) 8,128,024 7,483,986
Ehcache2_Lru 4,205,936 4,697,745
Ehcache3_Lru 10,051,020 13,939,317
Infinispan_Old_Lru 7,538,859 7,332,973
Infinispan_New_Lru 4,797,502 5,086,305