Simple Docker Image for Dynomite.
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
.gitignore
Dockerfile
LICENSE
README.md
Vagrantfile
dynomite-docker-mac.sh
dynomite-docker.sh
redis.conf
redis_cluster_1.yml
redis_cluster_2.yml
redis_cluster_21.yml
redis_cluster_22.yml
redis_cluster_23.yml
redis_cluster_3.yml
redis_cluster_6S1.yml
redis_cluster_6S2.yml
redis_cluster_6S3.yml
redis_cluster_6S4.yml
redis_cluster_6S5.yml
redis_cluster_6S6.yml
start.sh

README.md

dynomite-docker

Simple Docker Image for Dynomite. Dynomite-docker provides utilities to create clusters(3 node cluster), get seeds, test cluster health and replication via DCC.

Linux: How to use it? (native)

  1. Download and instal Java 8 -> http://www.oracle.com/technetwork/java/javase/downloads/index.html
  2. Download and install Docker. -> https://docs.docker.com/engine/installation/
  3. Bake docker images $ ./dynomite-docker.sh bake
  4. Create the Dynomite clusters $ ./dynomite-docker.sh run_single 0.5.8

Windows/Mac: How to use it?

(MAC) Docker (Require changes on bash script)

  1. Install docker -> https://docs.docker.com/docker-for-mac/install/
  2. (just 1 time) Bake docker images $ sudo ./dynomite-docker-mac.sh bake
  3. Create the Dynomite clusters $ sudo ./dynomite-docker-mac.sh run_single 0.5.8

Vagrant (native)

  1. Download and instal Virtual Box 5 -> https://www.virtualbox.org/wiki/Downloads
  2. Download and install Vagrant -> https://www.vagrantup.com/downloads.html
  3. vagrant up
  4. vagrant ssh
  5. Create the Dynomite clusters $ cd dynomite-docker/ && sudo ./dynomite-docker.sh run_single 0.5.8

What about my data?

You can dump from ypou production cluster and import it with redis-cli and redis-dump.
More information here: https://gist.github.com/diegopacheco/6c1862553a8b35a2680f914a2e08accc

What dynomite versions are suppoorted?

  • 0.5.7-14
  • 0.5.8-5
  • 0.5.9-2
  • 0.6.0

What parameters can I use?

$ ./dynomite-docker.sh help
dynomite-docker: easy setup for dynomite clusters for development. Created by: Diego Pacheco.
functions:

bake        : Bakes docker image
run         : Run Dynomite docker 2 clusters for dual write
run_single  : Run Dynomite docker Single cluster
run_shard   : Run Dynomite docker Shard cluster
dcc         : Run Dynomite Cluster Checker for 2 clusters
dcc_single  : Run Dynomite Cluster Checker for single cluster
dcc_shard   : Run Dynomite Cluster Checker for shard cluster
info        : Get Seeds, IPs and topologies(all 3 possible clusters)
log         : Print dynomite logs, you need pass the node number. i.e: ./dynomite-docker log 1
cli         : Enters redis-cli on dynomite port. i.e: ./dynomite-docker cli 1
keys_single : Runs KEYS * command in all nodes(Single Cluster)
keys_shard  : Runs KEYS * command in all nodes(Shard Cluster)
stop        : Stop and clean up all docker running images
help        : help documentation

How it works?

  1. We bake a docker image with Dynomiete v0.5.X and Redis 3.x.
  2. We create 2 clusters - each cluster has 3 nodes.
  3. In the end of the script(./dynomite-docker.sh run DYNOMITE_VERSION) you will see all seeds(We also run Dynomite Cluster Checker)
  4. You just need run ./dynomite-docker.sh bake 1 time.
  5. You can run ./dynomite-docker.sh run as many times as you want. First thing on the script we delete old docker images and old docker network - so we create new docker images and network every time you run the script create-dynomite-docker-cluster.sh.

What are my seeds?

Cluster 1A

179.18.0.101:8101:rack1:dc:100|179.18.0.102:8101:rack2:dc:100|179.18.0.103:8101:rack3:dc:100

Cluster 1B

179.18.0.201:8101:rack1:dc:100|179.18.0.202:8101:rack2:dc:100|179.18.0.203:8101:rack3:dc:100

Cluster 2 - Single

179.18.0.101:8101:rack1:dc:100|179.18.0.102:8101:rack2:dc:100|179.18.0.103:8101:rack3:dc:100

Cluster 3 - Shard

179.18.0.101:8102:rack1:dc:1383429731|179.18.0.102:8102:rack2:dc:1383429731|179.18.0.103:8102:rack3:dc:1383429731|179.18.0.104:8102:rack1:dc:2815085496|179.18.0.105:8102:rack2:dc:2815085496|179.18.0.106:8102:rack3:dc:2815085496

What's is the cluster Topology?

Summary

Cluster 1(1A and 1B): This cluster is create by command run - This is ideal for Dual write tests.
Cluster 2(Single) : This cluster is create by command run_single - This is ideal dev tests.
Cluster 3(Shard) : This cluster is create by command run_shard - This is ideal dev sharding keys tests.

Topologies

Cluster 1A

node: 1 - ip: 179.18.0.101 - Tokens: 100 - Rack: rack1 - DC: dc
node: 2 - ip: 179.18.0.102 - Tokens: 100 - Rack: rack2 - DC: dc
node: 3 - ip: 179.18.0.103 - Tokens: 100 - Rack: rack3 - DC: dc

Cluster 1B

node: 1 - ip: 179.18.0.201 - Tokens: 100 - Rack: rack1 - DC: dc
node: 2 - ip: 179.18.0.202 - Tokens: 100 - Rack: rack2 - DC: dc
node: 3 - ip: 179.18.0.203 - Tokens: 100 - Rack: rack3 - DC: dc

Cluster 2 - Single

node: 1 - ip: 179.18.0.101 - Tokens: 100 - Rack: rack1 - DC: dc
node: 2 - ip: 179.18.0.102 - Tokens: 100 - Rack: rack2 - DC: dc
node: 3 - ip: 179.18.0.103 - Tokens: 100 - Rack: rack3 - DC: dc

Cluster 3 - Shard

node: 1 - ip: 179.18.0.101 - Tokens: 1383429731 - Rack: rack1 - DC: dc
node: 2 - ip: 179.18.0.102 - Tokens: 1383429731 - Rack: rack2 - DC: dc
node: 3 - ip: 179.18.0.103 - Tokens: 1383429731 - Rack: rack3 - DC: dc
node: 4 - ip: 179.18.0.104 - Tokens: 2815085496 - Rack: rack1 - DC: dc
node: 5 - ip: 179.18.0.105 - Tokens: 2815085496 - Rack: rack2 - DC: dc
node: 6 - ip: 179.18.0.106 - Tokens: 2815085496 - Rack: rack3 - DC: dc

Integrated with DCC checks.

This scritps will run DCC(https://github.com/diegopacheco/dynomite-cluster-checker). You should see something like this.

**** BEGIN DYNOMITE CLUSTER CHECKER ****
1. Checking cluster connection...
    OK - All nodes are accessible!
2. Checking cluster data replication...
SEEDS: [179.18.0.101:8101:rack1:dc:100, 179.18.0.102:8101:rack2:dc:100, 179.18.0.103:8101:rack3:dc:100]
Checking Node: 179.18.0.101
  TIME to   Insert DCC_dynomite_123_kt - Value: DCC_replication_works: 6.0 ms - 0 s
  TIME to   Get: DCC_dynomite_123_kt : 4.0 ms - 0 s
  200 OK - set/get working fine!
Checking Node: 179.18.0.102
  TIME to   Get: DCC_dynomite_123_kt : 3.0 ms - 0 s
  200 OK - set/get working fine!
Checking Node: 179.18.0.103
  TIME to   Get: DCC_dynomite_123_kt : 3.0 ms - 0 s
  200 OK - set/get working fine!
3. Checking cluster failover...
All Seeds Cluster Failover test: OK
4. Results as JSON...
{
 "timeToRun": "3 seconds",
 "failoverStatus": "OK",
 "badNodes": [],
 "nodesReport":
[
  {
    "server":"179.18.0.101",
    "seeds":"[179.18.0.101:8101:rack1:dc:100, 179.18.0.102:8101:rack2:dc:100, 179.18.0.103:8101:rack3:dc:100]",
    "insertTime":"6.0 ms",
    "getTime":"4.0 ms",
    "consistency":"true"
  },
  {
    "server":"179.18.0.102",
    "getTime":"3.0 ms",
    "consistency":"true"
  },
  {
    "server":"179.18.0.103",
    "getTime":"3.0 ms",
    "consistency":"true"
  }
]
}

**** END DYNOMITE CLUSTER CHECKER ****
**** BEGIN DYNOMITE CLUSTER CHECKER ****

1. Checking cluster connection...
    OK - All nodes are accessible!
2. Checking cluster data replication...
SEEDS: [179.18.0.201:8101:rack1:dc:100, 179.18.0.202:8101:rack2:dc:100, 179.18.0.203:8101:rack3:dc:100]
Checking Node: 179.18.0.201
  TIME to   Insert DCC_dynomite_123_kt - Value: DCC_replication_works: 5.0 ms - 0 s
  TIME to   Get: DCC_dynomite_123_kt : 2.0 ms - 0 s
  200 OK - set/get working fine!
Checking Node: 179.18.0.202
  TIME to   Get: DCC_dynomite_123_kt : 2.0 ms - 0 s
  200 OK - set/get working fine!
Checking Node: 179.18.0.203
  TIME to   Get: DCC_dynomite_123_kt : 2.0 ms - 0 s
  200 OK - set/get working fine!
3. Checking cluster failover...
All Seeds Cluster Failover test: OK
4. Results as JSON...
{
 "timeToRun": "2 seconds",
 "failoverStatus": "OK",
 "badNodes": [],
 "nodesReport":
[
  {
    "server":"179.18.0.201",
    "seeds":"[179.18.0.201:8101:rack1:dc:100, 179.18.0.202:8101:rack2:dc:100, 179.18.0.203:8101:rack3:dc:100]",
    "insertTime":"5.0 ms",
    "getTime":"2.0 ms",
    "consistency":"true"                                                                  },

  {
    "server":"179.18.0.202",
    "getTime":"2.0 ms",
    "consistency":"true"
  },
  {
    "server":"179.18.0.203",
    "getTime":"2.0 ms",
    "consistency":"true"
  }
]
}


**** END DYNOMITE CLUSTER CHECKER ****

Similar projects