Skip to content

Docker & VoltDB Clustering Intro

Jump to bottom
John Hugg edited this page Apr 12, 2016 · 3 revisions

Prerequisites

Ensure you've installed JDK8, Docker and boot2docker (if on a Mac).

http://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html https://docs.docker.com/installation/mac/ https://docs.docker.com/installation/ubuntulinux/

Download VoltDB: http://learn.voltdb.com/DLSoftwareDownload.html

VoltDB Docker Hello World

The first step is to create a folder in which to build our VoltDB image.

mkdir voltdb-docker; cd voltdb-docker

Next, unpack your downloaded VoltDB tar.gz file into this directory in a folder named "voltdb-ent".

tar -xzf /path/to/voltdb-ent-6.2.tar.gz voltdb-ent

Create a file named "Dockerfile" in the directory with the following contents:

# VoltDB on top of Docker base JDK8 images
FROM java:8
WORKDIR /root
COPY voltdb-ent/ voltdb-ent/
WORKDIR /root/voltdb-ent
CMD /bin/bash

Now build the Docker image:

docker build -t yourname/voltdb .

Start your Docker image:

docker run --name=voltdb -t -i yourname/voltdb

This will start a container based on the image you built and drop you into a shell inside the voltdb-ent directory, ready to run VoltDB. To run, just type:

bin/voltdb create

You've now run VoltDB in a Docker Container. That's very cool, but it's not tremendously more useful than running VoltDB on your host. Let's stop and clear this container and restart with some improvements. To stop VoltDB and the container:

  1. Type ctrl-c to quit VoltDB.
  2. Type exit to stop the container and return to your regular shell prompt.
  3. Type docker rm voltdb to remove the container.

Improvements

Make the VoltDB Process the Container's Entry Point

Try starting the container using a command to start VoltDB:

docker run --name=voltdb -t -i yourname/voltdb /root/voltdb-ent/bin/voltdb create

Now you can exit with one fewer step:

  1. Type ctrl-c to quit VoltDB and stop the container.
  2. Type docker rm voltdb to remove the container.

Expose Management Port

docker run --name=voltdb -t -i -p 8080:8080 yourname/voltdb /root/voltdb-ent/bin/voltdb create

Now you can view the VoltDB Management Console by visiting:

http://localhost:8080/

Note that if you're using boot2docker, you may have to set up VirtualBox to also forward port 8080 from the boot2docker machine to port 8080 on your local machine. This double-forwarding is silly, but it works.

Run the Docker Container in the Background

Change the docker run ... command to use -d for "daemon", instead of -t -i for "tty" and "interactive" like so:

docker run --name=voltdb -d -p 8080:8080 yourname/voltdb /root/voltdb-ent/bin/voltdb create

Note that you now have to stop the node instead of typing ctrl-c. New shutdown steps:

  1. Type docker stop voltdb to quit VoltDB and stop the container.
  2. Type docker rm voltdb to remove the container.

Add a Hostname

If you don't want your hostname to be something like e17cc9a88500..., then set the hostname on the command line:

docker run --name=voltdb --hostname=volt1 -d -p 8080:8080 yourname/voltdb /root/voltdb-ent/bin/voltdb create

Now the Volt Management Console will refer to this node at "volt1" which is easier to see.

Moving to a 3-Node Cluster

Automating Deployment Files

Until this point, we've been using the default deployment file you get when you start VoltDB using voltdb create. If we want to use 3 nodes, we're going to need a non-default deployment file. We could embed a fixed, 3-node deployment file in the Docker image, but that would be annoyingly non-general. So instead we're going to do something lame and practical; we're going to add a script that generates a deployment file and starts VoltDB with it.

Back in the directory that contains our dockerfile, add this script as a new file named deploy.py.

#!/usr/bin/env python

import sys, os

deploymentText = """<?xml version="1.0"?>
<deployment>
    <cluster hostcount="##HOSTCOUNT##" kfactor="##K##" />
    <httpd enabled="true"><jsonapi enabled="true" /></httpd>
</deployment>
"""

deploymentText = deploymentText.replace("##HOSTCOUNT##", sys.argv[1])
deploymentText = deploymentText.replace("##K##", sys.argv[2])

with open('deployment.xml', 'w') as f:
	f.write(deploymentText)

os.execv("/root/voltdb-ent/bin/voltdb", 
         ["voltdb", 
          "create", 
          "--deployment=deployment.xml", 
          "--host=" + sys.argv[3]])

Make sure this script is executable. Try something like chmod a+x deploy.py to accomplish this.

Now update our dockerfile to copy the new script in:

# VoltDB on top of Docker base JDK8 images
FROM java:8
WORKDIR /root
COPY voltdb-ent/ voltdb-ent/
COPY deploy.py voltdb-ent/       # this is a new line
WORKDIR /root/voltdb-ent
CMD /bin/bash

Rebuild the docker image using:

docker build -t yourname/voltdb .

And now we can start a 1 node (k=0) VoltDB cluster this way:

docker run --name=voltdb --hostname=volt1 -d -p 8080:8080 \ yourname/voltdb /root/voltdb-ent/deploy.py 1 0 localhost

Networking for a 3-Node Cluster

Hypothetically, we could start a 3 node (k=1) cluster now, using the following plan:

docker run --name=volt1 --hostname=volt1 -d -p 8080:8080 \
    yourname/voltdb /root/voltdb-ent/deploy.py 3 1 volt1

Find the IP of the first container using:

docker inspect --format '{{ .NetworkSettings.IPAddress }}' voltdb`

Then use that IP as the leader for the second and third nodes:

docker run --name=volt2 --hostname=volt2 -d -p 8081:8080 \
    yourname/voltdb /root/voltdb-ent/deploy.py 3 1 $LEADERIP
docker run --name=volt3 --hostname=volt3 -d -p 8082:8080 \
    yourname/voltdb /root/voltdb-ent/deploy.py 3 1 $LEADERIP

Note that the ports for the next two node's management consoles are mapped to 8081 and 8082. Again, in boot2docker, these will need to be remapped into the local machine using VirtualBox.

But isn't that middle step a bit annoying. Can we eliminate it? We just need to use container network linking. What that does is add an entry in one running container that references another running container. Using linking, we can now start a 3 node cluster in 3 steps.

docker run --name=v1 --hostname=volt1 -d -p 8080:8080 \
    yourname/voltdb /root/voltdb-ent/deploy.py 3 1 volt1
docker run --name=v2 --hostname=volt2 --link=volt1:volt1 -d -p 8081:8080 \
    yourname/voltdb /root/voltdb-ent/deploy.py 3 1 volt1
docker run --name=v3 --hostname=volt3 --link=volt1:volt1 -d -p 8082:8080 \
    yourname/voltdb /root/voltdb-ent/deploy.py 3 1 volt1
Clone this wiki locally