Skip to content
This repository

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Test bed for big architecture using vagrant and puppet.

branch: master

Fetching latest commit…

Octocat-spinner-32-eaf2f5

Cannot retrieve the latest commit at this time

Octocat-spinner-32 manifests
Octocat-spinner-32 .gitignore
Octocat-spinner-32 README.org
Octocat-spinner-32 Rakefile
Octocat-spinner-32 Vagrantfile
Octocat-spinner-32 network.yml
README.org

Cluster eHaelix

Description

The goal is to mimic a eNovance environment in order to improve the continuous integration of specific development and puppet recipe.

The setup is based on virtualbox driven by vagrant and a specific development (machine-nics) which setups the host virtual networking environment.

The environment is composed of:

  • a cluster eHaelix;
  • a cluster of firewall;
  • two machines providing mx and dns services;
  • a monitoring machine;
  • a puppet master;
Type machine ip service done
ehaelix b1 172.143.115.36 openvz
drbd
b2 172.143.115.37 openvz
drbd
fw fw1 gateway
iptables
heartbeat1.3
pacemaker
fw2 gateway
iptables
heartbeat1.3
pacemaker
monitoring mon_1 centreon
nagios
graph
mon_2 centreon
nagios
graph
mail/dns dns_mx_1 dns
mx
dns_mx_2 dns
mx
backup backup_1 backuppc
backup_2 backuppc
outsider outside_1 no service

Eleven machines that create a realist production environment. Network bond and vlan are working inside each vm to mimic even more the production.

Not all machines need to be started for all needs. Only one firewall is required for the machine to get network connectivity. After if you test a puppet/backup recipe, you need only:

  • fw1;
  • backup_1;
  • backup_2;
  • puppet;

One machine consume roughly 475M of memory, but two machines do not make 950M. It roughly stay (> 500M) the same with no load on the machine. The maximum memory consumption is yet not know. Try it.

Setup

VirtualBox

A version above 4.0.0 is required.

For mainstream Linux use the packages available here.

For FreeBSD you can use redports virtualbox.

As a side note you need dkms on the host before installing virtualbox.

apt-get install dkms
/etc/init.d/virtualbox setup

Ruby

Requirements

You need :

  1. a version of the ruby interpreter above 1.9
  2. a version of the RubyGem above 1.8
  3. a version of bundler above 1.0

It is recommended to use native ruby tool for installation of the gem.

Here’s a example for Debian.

Debian example

TODO: test http://deb.bearstech.com/squeeze/ruby-1.9.3/

 sudo apt-get install ruby1.9.1
 sudo apt-get install ruby1.9.1-dev
 sudo apt-get install ri1.9
 
 # required to get gem >= 1.8 the 1.3 does not work.
 sudo update-alternatives --install /usr/bin/ruby ruby /usr/bin/ruby1.9.1 500 \
                          --slave   /usr/share/man/man1/ruby.1.gz ruby.1.gz /usr/share/man/man1/ruby1.9.1.1.gz \
                          --slave   /usr/share/man/man1/ri.1.gz ri.1.gz /usr/share/man/man1/ri1.9.1.1.gz \
                          --slave   /usr/share/man/man1/rdoc.1.gz rdoc.1.gz /usr/share/man/man1/rdoc1.9.1.1.gz \
                          --slave   /usr/share/man/man1/irb.1.gz irb.1.gz /usr/share/man/man1/irb1.9.1.1.gz \
                          --slave   /usr/bin/ri ri /usr/bin/ri1.9.1 \
                          --slave   /usr/bin/irb irb /usr/bin/irb1.9.1 \
                          --slave   /usr/bin/gem gem /usr/bin/gem1.9.1 \
                          --slave   /usr/bin/rdoc rdoc /usr/bin/rdoc1.9.1
 
 sudo env REALLY_GEM_UPDATE_SYSTEM=1 gem1.9.1 update --system 
 
 # must be greater than 1.8
 gem --version
 
 sudo gem install bundler -v '~>1.0'
 sudo gem install rake -v '~>0.9'

Vagrant

You need a specific version.

 cd /usr/src
 sudo git clone --branch vendor/sathlan git://github.com/sathlan/vagrant.git
 cd vagrant
 bundle install
 rake build
 sudo gem install pkg/vagrant-*.gem

Machine-nics

 apt-get install sudo
 # this step is very optional
 adduser vagrant
 # add sudo passwordless right to the user who spawn the vm
 echo '%vagrant ALL=(ALL) NOPASSWD: ALL' >> /etc/sudoers
 cd /usr/src
 git clone git://github.com/sathlan/machine-nics.git
 cd machine-nics
 bundle install
 rake build
 sudo gem install pkg/machine-nics-*.gem

cluster_healix

First we need the vagrant configuration and the associated rakefile (= makefile in ruby):

 su - vagrant                  # or whatever user belonging to the
                               # sudo nopasswd group
 cd ~/vps
 mkdir ~/cluster_ehaelix
 cd ~/cluster_ehaelix
 git clone git://github.com/sathlan/cluster.git .

We are done with dependencies. Now we can spawn some vm.

Basic Usage

First we need some network on the host.

 # list available network configuration
 rake -T
 # make one.
 rake add_enovance_small

Here you’ll have to make sure that everything which goes out is SNATed.

Finally we start the virtual machines. The first time is very long as it has to download the host (3.2G base PXE ehaelix type server) and create an vm out of it.

 vagrant up fw1
 vagrant up b1
 vagrant up puppet

Done, now you can enjoy the environment.

 vagrant help
 vagrant ssh b1

Snapshots

We can have virtualbox snapshot at the command line for free with vagrant-snap.

 sudo gem install virtualbox
 sudo gem install vagrant-snap
 cd ~/vps/cluster_ehaelix
 vagrant snap take b1 -n 'before big change' -d "I'm about to break everything."  

Basic Puppet integrated.

Vagrant has puppet provisioning integrated. Here, it’s use to setup network connectivity. Another use can be to quickly test an puppet module. Coupled with snapshot, it enable the user to get test without hurting an new module.

The basic setup is found in for each machine in the manifests/<machine>.pp file.

Problems

Linux

The network does not work

Setting up all the interface with nicpromis allow-all into virtualbox seems to resolve the issue. Seems related to Ticket VituralBox 4214.

On the host:

VBoxManage list vms  
for i in 2 3 4 5 ; do VBoxManage modifyvm <uuid> --nicpromisc$i allow-all; done

TODO: can be integrated to vagrant.

The network does not work - bis

The ifenslave is required for the puppet network configuration I made. Unfortunately, it does not come by default with the ehaelix PXE setup.

So :

apt-get update
apt-get install ifenslave
hostname | reboot

TODO: include it in the default ehaelix image. TODO: make a rule in puppet.

The gateway does not work for the FW

The setup has some glitch under Linux, but in the end everything works fine.

To have the bond0.101 works on the fw1, the bond0 interface must be set promisc mode:

ifconfig bond0 promisc

This seems also related to Ticket VituralBox 4214 and requires further investigation (or a puppet rule)

Common

By default the vagrant link (with 10….) address is still there. It makes the command vagrant ssh b1 work. But it add a default route. This must be (manually) removed.

TODO: A puppet rule to make it disappear.

Something went wrong with that request. Please try again.