Multi Node Setup with Raspberry PIs
One of the design goals for Xenon is the ability to run in low-cost, memory-constrained environments. To verify this, we have setup a Xenon cluster on 5 Raspberry PIs at VMware that is integrated with Xenon CI and gets used to run Xenon multi-node tests.
This article documents the steps involved in setting up a Raspberry PI 3 to run Xenon:
You can find detailed instructions for how to install the Operating System from Raspberry's website Link. For our 5 node setup, we chose Raspbian Jessie Link. OS installation on a Raspberry PI only involves setting up the micro-SD card with the OS image and should take a few minutes depending on the OS you are installing.
Make sure it is connected to a network with internet connectivity. After booting the Pi, change basic settings including Locale, Timezone, Password.
You can rely on DHCP for the raspberry Pi. However, DHCP ip addresses can change that can cause issues when running Xenon as part of a cluster. We recommend you instead use static-ip addresses on the raspberry pi in such scenarios. Raspbian Jessie uses dhcpcd5 by default. To set a static-ip address for dhcpcd5, you can do the following: Edit /etc/dhcpcd.conf and add the below lines at the end of the configuraiton file:
interface eth0
static ip_address=10.118.96.72/23
static routers=10.118.97.253
static domain_name_servers=10.118.97.253
Once this is done, delete the below two files and reboot the raspberry pi:
- /var/lib/dhcp/dhclient.leases
- /var/lib/dhcpcd5/dhcpcd-eth0.lease
When running a Xenon cluster, it is important that all nodes are within some time epsilon (order of minutes) in respect to wall clock time. The replication and consensus does not rely on timestamps, but higher level validation methods, do use time difference between nodes as an additional sanity check on node stability and availability.
Since a Raspberry Pi does not have a system clock, you will have to rely on NTP or something similar to keep the time updated on the PI. Raspbian Jessie already has NTP installed on it. In-addition we also installed ntpdate through: apt-get install ntpdate. NTP relies on UDP port 123, so make sure the port is available on your network and is not blocked by Firewall. If the port is blocked, NTP will not will be able to update the time locally. Also, update the ntp pool servers appropriately based on your location. This can be done by updating the file under /etc/ntp.conf. For US, the servers should be set to:
- server 0.us.pool.ntp.org
- server 1.us.pool.ntp.org
- server 2.us.pool.ntp.org
- server 3.us.pool.ntp.org
Once the pools have been updated, you will restart the ntp service through /etc/init.d/ntp restart.
Raspbian Jessie by default contains JRE/JDK. Once the above changes have been made, the Raspberry Pi should be ready to run your Xenon-Host. You can copy your Xenon jars manually and kick-off the host. For convenience, we have added some bash scripts: cluster-setup, cluster-cleanup that will deploy and start the xenon-host jars.