Skip to content

Vagrant

Jump to bottom Edit New page
Roemer Bakker edited this page Jan 15, 2018 · 106 revisions

Overview

Vagrant is a software that creates, and configures virtual development environments, by acting as a wrapper for virtualization applications such as:

More specifically, Vagrant allows virtual environments to be recreated in a repeatable fashion. This is useful when applications are shared between developers, and environment consistency is needed.

Additionally, Vagrant supports various configuration management (CM) applications to provision more granular system configurations. The following are a few CM applications that Vagrant supports:

Lastly, tools such as Jenkins, or Travis can be incorporated within a Vagrant box, which allow automated unit testing. This can be useful, if automated unit testing is incorporated into pull requests for versioning systems like github.

Note: when examples are required, this wiki will use Puppet as the configuration management application, and VirtualBox as the virtualization application.

Installation

One requirement of Vagrant, is the implementation of a virtualization application. As noted earlier, VirtualBox will be chosen. Therefore, the following shows how to install the required components for Vagrant on a debian (i.e. Ubuntu host) based machine:

# VirtualBox (with extension pack)
sudo apt-get install virtualbox
sudo apt-get install virtualbox-dkms
wget http://download.virtualbox.org/virtualbox/4.3.10/Oracle_VM_VirtualBox_Extension_Pack-4.3.10-93012.vbox-extpack

# add host user to 'vboxusers' group
sudo usermod -a -G vboxusers jeffrey

# Vagrant
wget https://dl.bintray.com/mitchellh/vagrant/vagrant_1.7.2_x86_64.deb
sudo dpkg --install vagrant_1.7.2_x86_64.deb

Note: change jeffrey to the current user on the host machine. The usermod definition allows the VirtualBox extension pack to implement USB integration between the host, and guest machine.

Note: when implementing an extension pack, the installed version of VirtualBox must match the extension pack version. This means, if VirtualBox 4.3.10 has been installed, the extension pack 4.3.10 must be installed. To check the installed version on the current local machine:

sudo VBoxManage list extpacks

Note: the dkms package provides support for installing supplementary versions of kernel modules, while virtualbox-dkms ensures that VirtualBox host kernel modules (vboxdrv, vboxnetflt, and vboxnetadp) are properly updated if the linux kernel version changes during the next apt-get upgrade.

The following will generally resolve kernel conflicts:

# resolve virtualbox kernel conflicts
sudo dpkg-reconfigure virtualbox-dkms
sudo dpkg-reconfigure virtualbox

# load vboxdrv module, and fix eth0
sudo modprobe vboxdrv
sudo modprobe vboxnetflt

Configuration

When implementing Vagrant to reproduce virtualization environments, there are generally two important components:

  • Boxes: the package format for vagrant environments.
  • Vagrantfile: describes the type of machine, and how to configure, and provision them.
    • USB Integration: allow USB devices to mount from the host machine to the virtual environment (guest)
  • Provisioning: generally optional, and useful for granular system configuration.

Once a vagrant box and been loaded (via vagrant up) into the virtualization application, the following commands can be implemented:

  • vagrant provision: provision a virtual environment defined in the Vagrantfile.
  • vagrant reload: start up the virtual environment defined in the Vagrantfile.
  • vagrant halt: shuts down the current virtual environment.
  • vagrant ssh <machine_name>: ssh into the desired virtual environment. When only a single virtual environment exists, the <machine_name> can be left off.
  • vagrant help, will list additional vagrant commands available.

However, on some host machines, when trying to run a virtual environment within a virtualization application (i.e. VirtualBox), an error similar to the following may appear:

Failed to open a session for the virtual machine xxxx.

VT-x is disabled in the BIOS (VERR_VMX_MSR_VMXON_DISABLED

By default, most machines have VT-x disabled. Therefore, to eliminate the above error, enter into the host machines bios (via reboot, and F10), and enter the System Configuration equivalent tab. Then, simply enable the Virtualization Technology (or equivalent), and reboot into the host machine.

Note: the first three commands require to be executed in a directory containing a Vagrantfile.

Boxes

Vagrant Boxes are the package format used within vagrant environments, and similar in concept to tarballs, or the zip file format. Specifically, a vagrant box is a virtual environment compressed into a standard format, using the .box file-extension. This allows virtual environments to be reloadable in a reliable, and consistent manner.

There are two ways to obtain vagrant boxes. The first involves using one already created. This can be as simple as searching from the Atlas public repository. However (the second), sometimes it is preferred to create a custom vagrant box.

Create Virtual Environment

When creating a VM, a vagrant user with the password of vagrant need to be created, to follow the vagrant convention. Then, dhcp needs to be configured before the virtual environment can be converted into a base box:

$ ip l
1: lo: <LOOPBACK,UP,LOWER_UP> mtu ***** qdisc noqueue state UNKNOWN mode DEFAULT
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eno********: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu **** qdisc pf ifo_fast state UP mode DEFAULT qlen ****
    link/ether 00:0*:**:**:**:** brd **:**:**:**:**:**
# release current IP (-r), obtain new IP for eno******** interface
$ dhclient -r && dhclient eno********

To make the above configurations persistent (on redhat 7.x), ONBOOT=no must be changed to ONBOOT=yes within /etc/sysconfig/network-scripts/ifcfg-eno********:

TYPE=Ethernet
BOOTPROTO=dhcp
DEFROUTE=yes
PEERDNS=yes
PEERROUTES=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_PEERDNS=yes
IPV6_PEERROUTES=yes
IPV6_FAILURE_FATAL=no
NAME=eno********
UUID=********-****-****-****-************
DEVICE=eno********
ONBOOT=yes

Now, ping 8.8.8.8 should return packets, and additional configurations can be made:

The consistent username, and password for the virtual environment, is the general practice for the vagrant implementation. This allows the vagrant box to be distributed publicly. However, if a more secure setup is desired, change to a secret username, and password. Also, since vagrant implements SSH for communication between the host, and guest (vagrant box), an insecure CA insecure key-pair is implemented, by default. To obtain greater security, the default key-pair should be changed.

The following Ubuntu example installation, is Vagrant compliant (public distribution):

Host Name: vagrant-ubuntu-14
Root Password: vagrant
Full Name: vagrant
User: vagrant
Password: vagrant
Encrypt your home directory? Select No
Partitioning method: Guided – use entire disk and set up LVM
When prompted which software to install, select only OpenSSH server (for now)
Select to install GRUB boot loader on the master boot record

Then, guest additions can be installed on the VirtualBox:

# ubuntu
sudo apt-get install virtualbox-guest-x11

However, it may be easier to install virtualbox guest via Vagrantfile:

...
  ## Variables (ruby syntax)
  required_plugins = %w(vagrant-vbguest)
  plugin_installed = false

  ## Install Vagrant Plugins
  required_plugins.each do |plugin|
    unless Vagrant.has_plugin? plugin
      system "vagrant plugin install #{plugin}"
      plugin_installed = true
    end
  end

  ## Restart Vagrant: if new plugin installed
  if plugin_installed == true
    exec "vagrant #{ARGV.join(' ')}"
  end
...

On Redhat machines, the guest additions image must be inserted, and can be done via the virtualbox window. Specifically, by clicking Devices > Insert Guest Additions CD image.... Then, the following commands need to be implemented, to ensure guest additions is properly installed, and loaded within the redhat VM:

# preliminary step
$ yum install kernel-devel-$(uname -r) bzip2 gcc
# ensure latest kernel is running
$ yum update kernel*
# mount guest additions
$ mkdir /media/VirtualBoxGuestAdditions
$ mount -r /dev/sr0 /media/VirtualBoxGuestAdditions
# add KERN_DIR environment variable
$ KERN_DIR=/usr/src/kernels/`uname -r`
$ export KERN_DIR
# install guest addition
$ cd /media/VirtualBoxGuestAdditions
$ ./VBoxLinuxAdditions.run
# start guest additions services
$ systemctl -a | grep box
$ sudo systemctl enable vboxadd-service.service
$ sudo systemctl enable vboxadd-x11.service
$ sudo systemctl enable vboxadd.service

Now, each time the virtual machine is started, guest additions should be properly loaded:

$ lsmod | grep -io vboxguest | xargs modinfo | grep -iw version
version:        5.0.8

This allows the guest machine to have the following additional features:

  • graphics acceleration (including mouse)
  • full screen (resizeable)
  • file sharing between the host, and the shared directory on the guest
  • usb integration

Next, to eliminate the need to type sudo, and the corresponding password, the vagrant user can be made sudoless by typing the following in the terminal console:

visudo

Then, adding the following to the end of the file:

vagrant ALL=(ALL) NOPASSWD:ALL

Since, vagrant communicates from the host, to the guest machine over SSH, communication requires a public, and private key. However, since vagrant boxes are generally distributed publicly, both keys need to be known by everyone. This means vagrant uses an insecure key-pair.

The following sets up the public key on the vagrant box (guest machine):

mkdir -p /home/vagrant/.ssh
wget --no-check-certificate https://raw.github.com/mitchellh/vagrant/master/keys/vagrant.pub -O /home/vagrant/.ssh/authorized_keys
# Ensure we have the correct permissions set
chmod 0700 /home/vagrant/.ssh
chmod 0600 /home/vagrant/.ssh/authorized_keys
chown -R vagrant /home/vagrant/.ssh

This will allow the vagrant box (containing the public key) to communicate with vagrant host (containing the insecure private key). As noted in the Vagrant documentation, the file permission changes (see above) are required for the SSH implementation.

For Redhat systems, remember to enable packages, update yum:

$ subscription-manager repos --enable=*
$ sudo yum update

And, finally to register the Redhat VM instance:

$ subscription-manager register --username [SUBSCRIPTION_USER] --password [SUBSCRIPTION_PASSWORD] 
$ subscription-manager attach --auto

Note: if an initial yum update yields the following transaction error on Redhat 7.1:

...
Transaction check error:
  file /usr/lib64/python2.7/site-packages/M2Crypto-0.21.1-py2.7.egg-info from in
stall of m2crypto-0.21.1.pulp-13.el7sat.x86_64 conflicts with file from package
m2crypto-0.21.1-15.el7.x86_64

Then, the following may be implemented, as a temporary fix, to allow the yum update command:

$ mv /usr/lib64/python2.7/site-packages/M2Crypto-0.21.1-py2.7.egg-info/ /usr/lib64/python2.7/site-packages/jl_M2Crypto-0.21.1-py2.7.egg-info.backup
$ yum update

To unregister a redhat VM instance:

$ subscription-manager remove --all
$ subscription-manager unregister
$ subscription-manager clean

Note: additional redhat commands may be reviewed.

Note: yum repos can be referenced from /etc/yum.repos.d.

Also, for a more secure setup, remove the above trusted public key, from ~/vagrant/.ssh/authorized_keys, create a custom key-pair, then configure the private key in the Vagrantfile with config.ssh.private_key_path, implement a different user, and password for the virtual environment.

Lastly, puppet must be installed:

# redhat / centos
$ sudo rpm -ivh https://yum.puppetlabs.com/puppetlabs-release-el-7.noarch.rpm
$ sudo yum install puppet
# ubuntu / debian
$ sudo apt-get install puppet
Create Base Box

Once the virtual environment is configured as desired (guest machine), creating a box can be accomplished within a terminal console of the host machine as follows:

vagrant package --base <VirtualBox_NAME>

which should create package.box in the current directory.

Note: replace <VirtualBox_NAME> with the virtual machine name (without brackets).

Implement Vagrant Box

A Vagrant box can be implemented either from an external source (i.e. Atlas), or from the local machine.

The following are some useful vagrant commands:

# create a vagrant base box
$ vagrant box add [box_reference] /path/to/package.box
# list all vagrant boxes
$ vagrant box list
# remove vagrant box
$ vagrant box remove [box_reference]

Note: the corresponding Vagrantfile can reference the vagrant box as follows:

...
  # Every Vagrant development environment requires a box. You can search for
  # boxes at https://atlas.hashicorp.com/search.
  config.vm.box = "[box_reference]"
...

The following implements mybox from the atlas user jeff1evesque:

cd [choose_directory]/
vagrant box add jeff1evesque/mybox
vagrant init --force --minimal jeff1evesque/mybox
vagrant up

The following implements a vagrant box from the Atlas public repository:

cd [choose_directory]/
vagrant box add ubuntu/trusty64
vagrant init --force --minimal ubuntu/trusty64
vagrant up

The following implements a vagrant box stored on the current local machine:

cd [choose_directory]/
vagrant add /path/to/mybox
vagrant init
vagrant up

It is important to know, running vagrant box add ... saves the vagrant box on the local machine. For linux based machines, the vagrant box gets stored in the ~/.vagrant.d/boxes directory, or the C:\Users\.vagrant.d\boxes for windows. Though, this is not so important, since Vagrant manages the boxes internally.

Next, running the vagrant init <box_name> command produces a Vagrantfile in the current working directory. The Vagrantfile contains the necessary configuration to load the desired virtual environment into a target virtualization application (i.e. VirtualBox). Specifically, the file is used with the command vagrant up. This means, if a vagrant box already has been downloaded, and a Vagrantfile already exists, then running the simple command vagrant up in the directory containing a Vagrantfile suffices to produce a desired virtual environment.

Vagrantfile

As discussed earlier, the Vagrantfile contains the configuration settings required to load a Vagrant Box as a virtual environment, within a desired virtualization application. Generally, if a vagrant box has previously been added (in ~/.vagrant.d/boxes), then the corresponding vagrant init command, will generate a default Vagrantfile. This file will contain the default configurations needed to create the desired virtual environment.

In most cases, the Vagrantfile is versioned (i.e. git). This allows developers to simply clone the repository (containing the Vagrantfile), add the corresponding vagrant box, and simply run vagrant up in the directory containing the Vagrantfile. When the virtual machine starts up, the directory containing the Vagrantfile on the host machine, gets mounted on the guest machine (loaded virtual machine), specifically in the /vagrant directory.

USB Integration

Before defining attributes in the Vagrantfile for persistent USB integration, the VBoxManage command can be used to attain information on USB devices currently mounted on the host machine:

$ sudo VBoxManage list usbhost
[sudo] password for jeffrey: 
Host USB Devices:

UUID:               xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
VendorId:           0x064e (064E)
ProductId:          0xc107 (C107)
Revision:           2.49 (0249)
Port:               4
USB version/speed:  2/2
Manufacturer:       SuYin
Product:            HP Webcam
SerialNumber:       xxxxx-xxxx-xxxx-xx-R02.43.01
Address:            sysfs:/sys/devices/pci0000:00/0000:00:1d.0/usb2/2-1/2-1.5//device:/dev/vboxusb/002/003
Current State:      Busy

UUID:               xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
VendorId:           0x0781 (0781)
ProductId:          0x5575 (5575)
Revision:           1.39 (0139)
Port:               0
USB version/speed:  2/2
Manufacturer:       SanDisk
Product:            Cruzer Glide
SerialNumber:       xxxxxxxxxxxxxxxxxxxx
Address:            sysfs:/sys/devices/pci0000:00/0000:00:1a.0/usb1/1-1/1-1.1//device:/dev/vboxusb/001/011
Current State:      Busy

Now, the Vagrantfile can implement the configuration attributes (determined above):

  # Enable USB Controller on VirtualBox
  config.vm.provider "virtualbox" do |vb|
    vb.customize ["modifyvm", :id, "--usb", "on"]
    vb.customize ["modifyvm", :id, "--usbehci", "on"]
  end

  # Implement determined configuration attributes
  config.vm.provider "virtualbox" do |vb|
    vb.customize ["usbfilter", "add", "0",
        "--target", :id,
        "--name", "Any Cruzer Glide",
        "--product", "Cruzer Glide"]
  end

Additionally, the attributes contained within the latter config.vm.provider can be adjusted (more can be defined) according to the above VBoxManage results. Specifically, the following are available properties that can be defined with respect to usbfilter:

  usbfilter                 add <index,0-N>
                            --target <uuid|vmname>|global
                            --name <string>
                            --action ignore|hold (global filters only)
                            [--active yes|no] (yes)
                            [--vendorid <XXXX>] (null)
                            [--productid <XXXX>] (null)
                            [--revision <IIFF>] (null)
                            [--manufacturer <string>] (null)
                            [--product <string>] (null)
                            [--remote yes|no] (null, VM filters only)
                            [--serialnumber <string>] (null)
                            [--maskedinterfaces <XXXXXXXX>]

Once this is defined, the USB device can be mounted on the guest machine upon vagrant halt, followed by vagrant up, or vagrant provision, followed by vagrant reload.

Note: Since USB devices may vary from time to time, it is difficult to predict ahead of time, which device should be defined within the Vagrantfile. Therefore, defining a USB filter via the VirtualBox GUI (Settings > USB > USB Device Filters), is many times preferred over the above Vagrantfile implementation.

Provisioning

Provisioning is the process of defining additional configurations after the virtual environment has been created by Vagrant, specifically the vagrant up command. Though, provisioning occurs after the virtual machine has been created, it is still associated with the vagrant up process.

Provisioning essentially involves implementing a provisioner (i.e. Puppet, Chef, Ansible, etc). This is done within the Vagrantfile, by defining the provisioning agent:

...

  # Custom Manifest: execute puppet logic in 'default.pp'. Specifically, this
  #                  file will install, and setup various configurations.
  config.vm.provision "puppet" do |puppet|
    puppet.manifests_path = "manifests"
    puppet.manifest_file = "default.pp"
  end
...

The above snippet within the Vagrantfile defines the provisioning application (puppet), the manifest directory (relative to the Vagrantfile), and the file containing the provisioning rules.

Lastly, if changes are made to the provisioning logic (either Vagrantfile, or the manifest file), the virtual machine should either be destroyed via vagrant destroy, and rebuilt using vagrant up, or reprovisioned. When implementing the former, it is recommended to let vagrant handle such process. This is generally preferred over manually deleting all files, related to the virtual environment, within the virtualization application. However, if reprovisioning is preferred, then simply execute vagrant provision, followed by the vagrant reload command in the terminal console.

Note: the above commands require the virtual environment to be active.

Port Forward

A virtual machine created via vagrant, can define any number of port forwarding rules, within the Vagrantfile.

The following implementation within the Vagrantfile:

Vagrant.configure(2) do |config|
...
  config.vm.network "forwarded_port", guest: 5000, host: 8080
  config.vm.network "forwarded_port", guest: 443, host: 8585
...
end

indicates port 5000 within the virtual machine (guest), can be accessed on port 8000 on the host machine. Similarly, port 443 is forwarded to port 8585, on the host machine.

Note: Installing vagrant-vbguest on the command line:

$ vagrant plugin install vagrant-vbguest

Alternatively, vagrant-vbguest can be installed as a plugin, within the Vagrantfile:

Vagrant.configure(2) do |config|
...
  ## Variables (ruby syntax)
  required_plugins = %w(vagrant-vbguest)
  plugin_installed = false

  ## Install Vagrant Plugins
  required_plugins.each do |plugin|
    unless Vagrant.has_plugin? plugin
      system "vagrant plugin install #{plugin}"
      plugin_installed = true
    end
  end
...
end

Either choice, will remove the following error traceback, during a vagrant up build:

...
Failed to mount folders in Linux guest. This is usually because
    the "vboxsf" file system is not available. Please verify that
    the guest additions are properly installed in the guest and
    can work properly. The command attempted was:

    mount -t vboxsf -o uid=`id -u vagrant`,gid=`getent group vagrant | cut -d: -f3` /vagrant /vagrant
    mount -t vboxsf -o uid=`id -u vagrant`,gid=`id -g vagrant` /vagrant /vagrant

  The error output from the last command was:

  /sbin/mount.vboxsf: mounting failed with the error: No such device

Note: the following syntax for the above Vagrantfile implementation, allow multiple vagrant plugins to be installed:

...
  required_plugins = %w(vagrant-plugin-1 vagrant-plugin-2 vagrant-vbguest)
...
Add a custom footer
Add a custom sidebar
Clone this wiki locally