This document describes how to spin up a multi-host Docker swarm cluster built on a VXLAN overlay network in AWS, where a host is a AWS instance and multiple hosts may reside in the same VPC or different VPCs. If there is underlying network connectivity to connect VPCs securely (such as using Aviatrix encrypted peering), a swarm cluster can span across multiple VPCs and to Azure and Google.
For a reference design on how to use Aviatrix OpenVPN® capability to remotely access containers in a swarm cluster in the same manner as accessing instances from your laptop, for example, being able to use “curl” to a container running a web service, check out this link.
To build a Swarm cluster, there needs to be a manager instance, a consul instance, and a few hosting instances. To simplify the topology, manager and consul are combined into one instance.
There are many How to resources online on creating a swarm cluster with a VXLAN overlay network, the guide below is intended to be a reference.
At AWS console, launch instances by using Amazon Linux AMI with Docker package, as shown below:
a. ssh into each instanceb. sudo yum updatec. curl -sSL https://get.docker.com/ | shd. sudo service docker starte. sudo service docker stopf. sudo docker daemon -H tcp://0.0.0.0:2375 -H unix:///var/run/docker.sock&g. sudo usermod -aG docker ec2-userh. logout
a. ssh into the manager/consul instanceb. ifconfig eth0 (to get the consul’s eth0 IP address, for example, 172.31.12.28)c. sudo docker run -d -p 8500:8500 --name=consul progrium/consul -server -bootstrapd. docker run -d -p 4000:4000 swarm manage -H :4000 --replication --advertise 172.31.12.28:4000 consul://172.31.12.28:8500
a. ssh into each instanceb. Follow the same procedures as described in Step 2 to install docker daemonc. sudo vi /etc/sysconfig/docker and add the following line:OPTIONS="--default-ulimit nofile=1024:4096 -H tcp://0.0.0.0:2375 -H unix:///var/run/docker.sock --cluster-advertise eth0:2375 --cluster-store consul://172.31.12.28:8500"d. sudo service docker restart
a. ssh into each instanceb. Use “ifconfig eth0” to get container hosting instance’s own IP address, e.g. 172.31.14.64Tell consul my IP address and to join the swarm cluster:sudo docker run -d swarm join --advertise=172.31.14.64:2375 consul://172.31.12.28:8500
On the manager/counsul instance:
a. ssh into each instanceb. Create an overlay network “my-overlay-network” with network CIDR “10.0.0.0/24”: docker -H :4000 network create - -subnet=10.0.0.0/24 my-overlay-networkc. To list the network and you will see the newly created “my-overlay-network” on each hosting instance joined the swarm clusterdocker network ls
On each container hosting instance:
a. ssh into the hostb. Launch a Ubuntu container “test01” within the overlay network “my-overlay-network:sudo docker run -itd --net my-overlay-network --name test01 ubuntu /bin/bashc. Find out the overlay IP address for container “test01” assigned by consul. There are at least three ways:
type command on the instance:
sudo docker inspect test01
The above command returns a json output, look for “IPAddress” under my-overlay-network.
Type command on the instance:
docker network inspect my-overlay-network, where my-overlay-network is the overlay network name.
Alternatively, use the following command to find out overlay IP address:
docker inspect -f '{{.Name}} - {{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' test01
where test01 is the container name.
Access into the container and install some network tools if you like:
sudo docker exec -ti test01 /bin/bashapt-get update --yesapt-get install net-tools --yesapt-get install iputils-ping –yesIf you like to have ssh access to your container, follow these steps:
apt-get install openssh-serverapt-get install vimsudo vi /etc/ssh/sshd_configand modify the following 2 lines to:PermitRootLogin yes#StrictModes yesSetup root password by typing command “passwd”Sudo service ssh restartifconfig eth0 to get the IP address- In the container, do “ping 10.0.0.x” to other containers you
created.
Note:
You may need to modify “Security Group” of each instance and manager to allow the access to their ports.
OpenVPN is a registered trademark of OpenVPN Inc.
.. disqus::