Yardstick Hazelcast is a set of Hazelcast Data Grid benchmarks written on top of Yardstick framework.
Visit Yardstick Repository for detailed information on how to run Yardstick benchmarks and how to generate graphs.
The documentation below describes configuration parameters in addition to standard Yardstick parameters.
- Create a local clone of Yardstick Hazelcast repository
- Import Yardstick Hazelcast POM file into your project
- Run
mvn package
command
The following benchmarks are provided:
HazelcastGetBenchmark
- benchmarks atomic distributed cache get operationHazelcastPutBenchmark
- benchmarks atomic distributed cache put operationHazelcastSetBenchmark
- benchmarks atomic distributed cache set operationHazelcastPutGetBenchmark
- benchmarks atomic distributed cache put and get operations togetherHazelcastSetGetBenchmark
- benchmarks atomic distributed cache set and get operations togetherHazelcastPutGetBatchBenchmark
- benchmarks atomic distributed cache set on many keys and getAll operations togetherHazelcastPutTxBenchmark
- benchmarks transactional distributed cache put operationHazelcastSetTxBenchmark
- benchmarks transactional distributed cache set operationHazelcastSetGetTxOptimisticBenchmark
- benchmarks transactional distributed cache set and get operations togetherHazelcastSetGetTxPessimisticBenchmark
- benchmarks transactional distributed cache set and getForUpdate operations togetherHazelcastSqlQueryBenchmark
- benchmarks distributed SQL query over cached dataHazelcastSqlQueryPutBenchmark
- benchmarks distributed SQL query with simultaneous cache updatesHazelcastSqlQuerySetBenchmark
- benchmarks distributed SQL query with simultaneous cache setsHazelcastPutAllBenchmark
- benchmarks atomic distributed cache putAll operationHazelcastSetAllTxBenchmark
- benchmarks transactional distributed cache set operations in one transactionHazelcastGetAllSetAllTxOptimisticBenchmark
- benchmarks transactional distributed cache set and get operations togetherHazelcastGetAllSetAllTxPessimisticBenchmark
- benchmarks transactional distributed cache set and getForUpdate operations togetherHazelcastAffinityCallBenchmark
- benchmarks affinity calls
All benchmarks extend HazelcastAbstractBenchmark
class. A new benchmark should also extend this abstract class and implement test
method. This is the method that is actually benchmarked.
Before running Hazelcast benchmarks, run mvn package
command. This command will compile the project and also will unpack scripts from yardstick-resources.zip
file to bin
directory.
Note that this section only describes configuration parameters specific to Hazelcast benchmarks, and not for Yardstick framework. To run Hazelcast benchmarks and generate graphs, you will need to run them using Yardstick framework scripts in
bin
folder.
Refer to Yardstick Documentation for common Yardstick properties and command line arguments for running Yardstick scripts.
The following Hazelcast benchmark properties can be defined in the benchmark configuration:
-nn <num>
or--nodeNumber <num>
- Number of nodes (automatically set inbenchmark.properties
), used to wait for the specified number of nodes to start-b <num>
or--backups <num>
- Number of backups for every key-hzcfg <path>
or--hzConfig <path>
- Path to Hazelcast configuration file-hzclicfg <path>
or--hzClientConfig <path>
- Path to Hazelcast client configuration file-sb
or--syncBackups
- Flag indicating whether synchronous backups are used, asynchronous is a default-nt
or--nodeType
- Type of driver nodes. One of:SERVER
,CLIENT
,LITE_MEMBER
-r <num>
or--range <num>
- Range of keys that are randomly generated for cache operations-rb
or--readBackups
- Flag indicating whether backup reads are enabled or not, disabled by default
For example if we need to run 2 HazelcastNode
servers on localhost with HazelcastPutBenchmark
benchmark on localhost, with number of backups set to 1, backups are synchronous, then the following configuration should be specified in benchmark.properties
file:
SERVER_HOSTS=localhost,localhost
# Note that -dn and -sn, which stand for data node and server node, are
# native Yardstick parameters and are documented in Yardstick framework.
CONFIGS="-b 1 -sb -dn HazelcastPutBenchmark -sn HazelcastNode"
This repo contains all necessary scripts and properties files for a comparison Hazelcast with other products.
For running on Amazon EC2 need to perform the following steps:
- Run Amazon EC2 instances. Choose number of instances and hardware according to your requirements.
The following actions need to perform on all instances:
- Install Java, Maven and Git on all instances.
For example for Ubuntu:
# apt-get install java
# apt-get install mvn
# apt-get install git
- Clone this repository and build project on one of the Amazon instances. Yardstick will copy all the built binaries on all nodes automatically.
git clone https://github.com/gridgain/yardstick-hazelcast
mvn clean package
-
Change
SERVER_HOSTS
andDRIVER_HOSTS
properties inconfig/benchmark.properties
file.SERVER_HOSTS
is comma-separated list of IP addresses where servers should be started, one server per host.DRIVER_HOSTS
is comma-separated list of IP addresses where drivers should be started, one driver per host, if the property is not defined then the driver will be run on localhost. Property file contains many useful information about benchmarks such aslist of benchmarks
,JVM opts
and etc. More details there Properties And Command Line Arguments -
Update IP addresses in network section from
config/hazelcast-config.xml
andconfig/hazelcast-client-config.xml
files. For example:
config/hazelcast-client-config.xml
...
<network>
<cluster-members>
<address>XXX.XXX.XXX.1:57500</address>
<address>XXX.XXX.XXX.2:57500</address>
<address>XXX.XXX.XXX.3:57500</address>
</cluster-members>
<connection-timeout>10000</connection-timeout>
<connection-attempt-limit>50</connection-attempt-limit>
</network>
...
config/hazelcast-config.xml
...
<network>
<port auto-increment="true">57500</port>
<join>
<multicast enabled="false"/>
<tcp-ip enabled="true">
<member>XXX.XXX.XXX.1:57500</member>
<member>XXX.XXX.XXX.2:57500</member>
<member>XXX.XXX.XXX.3:57500</member>
</tcp-ip>
</join>
</network>
...
- This repository contains two main set benchmark for sync and async backups (see more details about it there).
By default used set of benchmark for async backups. If you want to run benchmarks for sync backup then need to use
benchmark-sync.properties
file. Perform./bin/benchmark-run-all.sh
script for async backups or./bin/benchmark-run-all.sh ./config/benchmark-sync.properties
for sync backups. For more details about running scripts see Running Yardstick Benchmarks. - After execution the script in
result
folder will be saved to results of benchmarks. For visualisation of results can be usedbin/jfreechart-graph-gen.sh
script. For more details about the script see JFreeChart Graphs.
Use GitHub issues to file bugs.
Yardstick Hazelcast is available under Apache 2.0 Open Source license.