Vagrant Mesos (2 master/2 slave setup)
This Vagrant installation sets up a minimal Big Data Europe platform with two masters and two slaves. This installation can be used for getting a feel of the platform without a lengthy installation procedure.
The Big Data Europe platform provides a generic platform for deploying Big Data pipelines. The instances providing the computation are offered through docker images. See http://github.com/big-data-europe/ for examples.
You should ensure that you have the following available in order to use the platform:
- 10Gb RAM
- 30Gb Disk space
- VirtualBox - 5.0.10 (https://www.virtualbox.org)
- Vagrant - 1.7.4 (https://www.vagrantup.com/)
- CLI (Like GNU Bash on Unix systems, or cygwin on Windows)
Starting the cluster
- cd <clone directory>
- vagrant up
After first boot: restart the Vagrant instances to correct the network configuration.
- vagrant halt
- vagrant up
You can ssh into the machines using the regular Vagrant commands:
vagrant ssh slave1
The base cluster publishes the following nodes:
The Vagrantfile should be clear to understand and fairly easy to alter.
Playing with the cluster
The virtual cluster can be used for deploying applications and for testing the rigidity of the platform.
See the relevant applications on http://github.com/big-data-europe/ for more information on how to deploy applications on this cluster.
Trolling the cluster
Big Data clusters are one of the few machines which like to be trolled. Feeling frustrated, why don’t you kill a slave node? Need a bigger up, kill the master. Clustered systems should stay alive in these cases. This minimal setup allows you to play with that usecase.
- Visit http://192.168.0.10:5050/ which should show 2 actives slaves (shutting down a slave will leave to it becoming deactivated, etc.)
- Start the system, access the link and check that 2 slaves are visible
- Shut down one of the slaves (and watch it pass from deactivated to gone)
- Restart the slave and check it becomes active again
- Power down the master and check the other master continues
Starting optional Hadoop services
Hadoop services are not started on boot. In order to use HDFS or YARN, execute the following as hduser: