Skip to content
Switch branches/tags

Name already in use

A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch?
Go to file
Cannot retrieve contributors at this time

Redis Cluster Testing

Different tests to verify Redis Cluster functionality in terms of distributed systems theory:

  • CAP Theorem tests for consistency and availability during network partitions
  • session blocking tests


Setup a cluster of 3-4 nodes, for example on Digitalocean and create an Ansible inventory file

redis-test01 ansible_ssh_user=root ansible_ssh_host=IP1
redis-test02 ansible_ssh_user=root ansible_ssh_host=IP2
redis-test03 ansible_ssh_user=root ansible_ssh_host=IP3
redis-test04 ansible_ssh_user=root ansible_ssh_host=IP4






This creates a cluster with 3 redis nodes having 2 servers each and one web frontend server. The ansible groups [partition_a/b] control how iptables is used to split different servers from each other.

Provision this setup with:

$ ansible-playbook -i servers_inventory provision.yml

To make the cluster running, connect to all of the Redis nodes and call:

$ /root/redis-cluster start

To start all the servers on that node.

And then on one node create the cluster with redis-trib using:

$ /root/

You can use the file scripts/generate_user_urls.php to pass the ip/url to the web node and it will generate lists of urls to use with the tool siege. The traffic is designed such that it will hit all the shards of the cluster.

Generating load:

$ siege -f cluster_set.txt -c 100

You can monitor the cluster health by calling one of the following commands:

$ /var/www/redis-trib.rb info <HOST>:<PORT>
$ redis-cli -h <HOST> -p <PORT> cluster slots
$ redis-cli -h <HOST> -p <PORT> cluster nodes


You can modify the different config files in templates/ folder and reprovision the servers.

  • cluster.conf.j2 contains redis cluster related configuration
  • redis.conf.j2 contains redis server related configuration

Injecting faults

Network partition

You can partition the network with Ansible:

$ ansible-playbook -i servers_inventory partition.yml

You can heal the partition with:

$ ansible-playbook -i servers_inventory heal.yml

Kill Redis Node

Use redis-cli tool on one of the Redis nodes to call:

$ redis-cli -h IPADDRSES -p 7000 debug segfault

This forces a node to segfault and shows how the cluster automatically fails over. You can observe the faults happening during this process looking at the siege output.

Manual failover

Use redis-cli tool on one of the Redis nodes to call for a slave:

$ redis-cli -h IPADDRSES -p 7000 cluster failover