Apache Ignite on Docker 1.12 Swarm
This project builds on top of Apache Ignite to provide a Docker Cluster compatible image and service definition to automatically scale a simple
memcached compatible Ignite service across a cluster.
Apache Ignite exposes
memcached by default on port 11211. Note that only the binary protocol is supported.
memcached client documentation here for details
Building Java Package
Service is compiled to a single executable jar (
target/ignite-1.0.jar) by running:
Since the Maven
shade plugin is used to create an executable jar, the package can be run with
java -jar target/ignite-1.0.jar
Configuration as required by system properties (
Optionally initialize the default cache in full replication mode by passing the cache name as an argument.
This is useful to ensure the cache is configured as expected before connecting with a
Local Host Address
The Ignite local host address is normally discovered automatically from the first non-loopback address. This doesn't work very well in Docker, especially with an overlay network. Even if it selects the correct network interface, the VIP may be selected instead of the host address (both appear as addresses on the interface).
To aid in discovery, we can set a prefix which the address must match to be used (should match the overlay network address prefix). The matching addresses will be sorted numerically and the last address will be selected. This works well since the VIP tends to be the first address in the range.
Set the DNS address to resolve to discover the other IP addresses of nodes to connect with.
Docker Cluster provides a special DNS address
servicename) which solves to all hosts in the service cluster. Pass that DNS address as an argument to allow IP discovery.
Build Docker Image by running
docker build -t ignite .
Create a Docker cluster as described in ElasticSearch Cluster on Docker 1.12
docker service create \
--mode global \
--name myignite \
--network enc-net \
--publish 11211:11211 \
-DNETWORK_PREFIX=10.0.9. -DCLUSTER_DNS=tasks.myignite -DINIT_REPLICATE
--publish argument to restrict access to the overlay network.
For testing, the image can be manually transferred to the servers as follows.
(Use a Docker registry whenever possible)
docker save ignite > ignite.tar
pv ignite.tar | ssh -C manager1 sudo docker load
pv ignite.tar | ssh -C worker1 sudo docker load
pv ignite.tar | ssh -C worker2 sudo docker load
rm -f ignite.tar
The Ignite server only supports memcached binary protocol, so commands sent with
nc won't work.
Instead use the Java Memcached Binary Test Client