Benchmarks
JMH is used to build and run benchmarks. Benchmark results are obtained on a PC running under Windows 7 with Intel(R) Core(TM) i7-3770 3.4 GHz CPU and 64-bit JRE build 1.8.0_77-b03 with the following parameters: -Xms1g -Xmx1g. Nothing else is configured and tests run with default settings. To get results in your environment, run in the project root directory:
./gradlew testBenchmarksJMH
Tokyo Cabinet Benchmark is useful for comparing the performance of key/value storages. For one million 8-char string keys ('00000000', '00000001', etc.) and values equal to the keys, four operations are measured:
- writing all key/value pairs in ascending order;
- writing all key/value pairs in random order;
- reading all key/value pairs in ascending order;
- reading all key/value pairs in random order.
Currently, benchmark results are available for Xodus stores with and without key prefixing, Berkeley DB JE databases with and without key prefixing, MapDb tree map, and Chronicle Map. All the scores are in seconds per single benchmark run for one million keys. Excerpt of the output of the build running benchmarks is as follows:
# Run complete. Total time: 02:13:06
Benchmark Mode Cnt Score Error Units
chronicle.JMHChronicleMapTokyoCabinetReadBenchmark.randomRead ss 16 0.829 - 0.075 s/op
chronicle.JMHChronicleMapTokyoCabinetReadBenchmark.successiveRead ss 16 0.098 - 0.005 s/op
chronicle.JMHChronicleMapTokyoCabinetWriteBenchmark.randomWrite ss 16 1.238 - 0.133 s/op
chronicle.JMHChronicleMapTokyoCabinetWriteBenchmark.successiveWrite ss 16 1.184 - 0.099 s/op
env.JMHEnvTokyoCabinetReadBenchmark.randomRead ss 60 2.692 - 0.045 s/op
env.JMHEnvTokyoCabinetReadBenchmark.successiveRead ss 60 0.117 - 0.005 s/op
env.JMHEnvTokyoCabinetWriteBenchmark.randomWrite ss 60 2.208 - 0.046 s/op
env.JMHEnvTokyoCabinetWriteBenchmark.successiveWrite ss 60 0.699 - 0.018 s/op
env.JMHEnvWithPrefixingTokyoCabinetReadBenchmark.randomRead ss 60 1.289 - 0.020 s/op
env.JMHEnvWithPrefixingTokyoCabinetReadBenchmark.successiveRead ss 60 0.292 - 0.048 s/op
env.JMHEnvWithPrefixingTokyoCabinetWriteBenchmark.randomWrite ss 60 1.661 - 0.074 s/op
env.JMHEnvWithPrefixingTokyoCabinetWriteBenchmark.successiveWrite ss 60 0.777 - 0.064 s/op
je.JMH_JETokyoCabinetReadBenchmark.randomRead ss 60 3.208 - 0.027 s/op
je.JMH_JETokyoCabinetReadBenchmark.successiveRead ss 60 1.252 - 0.189 s/op
je.JMH_JETokyoCabinetWriteBenchmark.randomWrite ss 60 5.844 - 0.145 s/op
je.JMH_JETokyoCabinetWriteBenchmark.successiveWrite ss 60 3.441 - 0.187 s/op
je.JMH_JEWithPrefixingTokyoCabinetReadBenchmark.randomRead ss 60 3.504 - 0.070 s/op
je.JMH_JEWithPrefixingTokyoCabinetReadBenchmark.successiveRead ss 60 1.302 - 0.200 s/op
je.JMH_JEWithPrefixingTokyoCabinetWriteBenchmark.randomWrite ss 60 5.905 - 0.205 s/op
je.JMH_JEWithPrefixingTokyoCabinetWriteBenchmark.successiveWrite ss 60 3.411 - 0.209 s/op
mapdb.JMHMapDbTokyoCabinetReadBenchmark.randomRead ss 16 6.705 - 0.196 s/op
mapdb.JMHMapDbTokyoCabinetReadBenchmark.successiveRead ss 16 0.274 - 0.091 s/op
mapdb.JMHMapDbTokyoCabinetWriteBenchmark.randomWrite ss 16 21.822 - 0.380 s/op
mapdb.JMHMapDbTokyoCabinetWriteBenchmark.successiveWrite ss 16 26.588 - 0.517 s/op
The same results in table form:
| Random Read | Successive Read | Random Write | Successive Write | |
|---|---|---|---|---|
| Xodus store with key prefixing | 1.289 | 0.292 | 1.661 | 0.777 |
| Xodus store without key prefixing | 2.692 | 0.117 | 2.208 | 0.699 |
| BDB JE database with key prefixing | 3.504 | 1.302 | 5.905 | 3.411 |
| BDB JE database without key prefixing | 3.208 | 1.252 | 5.844 | 3.441 |
| MapDB tree map | 6.705 | 0.274 | 21.822 | 26.588 |
| Chronicle Map | 0.829 | 0.098 | 1.238 | 1.184 |