Benchmarks

Ben Manes edited this page Nov 9, 2015 · 30 revisions

The benchmarks use Java microbenchmark harness to provide an accurate analysis. The caches are configured as,

  • Caffeine and ConcurrentLinkedHashMap size their internal structures based on the number of CPUs
  • Guava was configured with a concurrency level of 64 (defaults to 4 to reduces memory usage). Note that Guava #2063 would resolve the poor performance, but has been backlogged for years (up to 25x faster!).
  • Ehcache v2 is internally hardcoded to 100 segments, whereas v3 is not segmented
  • Infinispan "old" is a Guava-like cache configured with a concurrency level of 64
  • Infinispan "new" is a rewrite using a lock-free deque (default in v7.2+)

Desktop-class

Run on a MacBook Pro i7-4870HQ CPU @ 2.50GHz (4 core) 16 GB Yosemite.

Compute

In this benchmark the caches are unbounded, fully populated, and the computation returns a constant value. This benchmark demonstrates the overhead due to locking when the entry is present. Caffeine performs a lock-free prescreening before falling back to an atomic compute if absent call. The scenarios graphed are all threads retrieving a single entry ("sameKey") and threads retrieving different keys based on a Zipf distribution ("spread").

Read (100%)

In this benchmark 8 threads concurrently read from a cache configured with a maximum size.

Read (75%) / Write (25%)

In this benchmark 6 threads concurrently read from and 2 threads write to a cache configured with a maximum size.

Write (100%)

In this benchmark 8 threads concurrently write to a cache configured with a maximum size.

Server-class

The benchmarks were run on an Azure G4 instance, the largest available during a free trial from the major cloud providers. The machine was reported as a single socket Xeon E5-2698B v3 @ 2.00GHz (16 core, hyperthreading disabled), 224 GB, Ubuntu 15.04.

Compute

Cache same key spread
ConcurrentHashMap 29,679,839 65,726,864
Caffeine 1,581,524,763 530,182,873
Guava 25,132,366 114,608,951

Read (100%)

Unbounded ops/s (8 threads) ops/s (16 threads)
ConcurrentHashMap (v8) 560,367,163 1,171,389,095
ConcurrentHashMap (v7) 301,331,240 542,304,172
Bounded
Caffeine 181,703,298 382,355,194
ConcurrentLinkedHashMap 154,771,582 313,892,223
LinkedHashMap_Lru 9,209,065 13,598,576
Guava (default) 12,434,655 10,647,238
Guava (64) 24,533,922 43,101,468
Ehcache2_Lru 11,252,172 20,750,543
Ehcache3_Lru 11,415,248 17,611,169
Infinispan_Old_Lru 29,073,439 49,719,833
Infinispan_New_Lru 4,888,027 4,749,506

Read (75%) / Write (25%)

Unbounded ops/s (8 threads) ops/s (16 threads)
ConcurrentHashMap (v8) 441,965,711 790,602,730
ConcurrentHashMap (v7) 196,215,481 346,479,582
Bounded
Caffeine 144,193,725 279,440,749
ConcurrentLinkedHashMap 63,968,369 122,342,605
LinkedHashMap_Lru 8,668,785 12,779,625
Guava (default) 11,782,063 11,886,673
Guava (64) 22,782,431 37,332,090
Ehcache2_Lru 9,472,810 8,471,016
Ehcache3_Lru 10,958,697 17,302,523
Infinispan_Old_Lru 22,663,359 37,270,102
Infinispan_New_Lru 4,753,313 4,885,061

Write (100%)

Unbounded ops/s (8 threads) ops/s (16 threads)
ConcurrentHashMap (v8) 60,477,550 50,591,346
ConcurrentHashMap (v7) 46,204,091 36,659,485
Bounded
Caffeine 55,281,751 48,295,360
ConcurrentLinkedHashMap 23,819,597 39,797,969
LinkedHashMap_Lru 10,179,891 10,859,549
Guava (default) 4,764,056 5,446,282
Guava (64) 8,128,024 7,483,986
Ehcache2_Lru 4,205,936 4,697,745
Ehcache3_Lru 10,051,020 13,939,317
Infinispan_Old_Lru 7,538,859 7,332,973
Infinispan_New_Lru 4,797,502 5,086,305
You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.
Press h to open a hovercard with more details.