LISA18: Test Driven Infrastructure

Introduction

Test Driven Infrastructure (Compliance as Code) is the process to design the infrastructure through tests.

The idea is that you would first experiment, model, and build out your infrastructure, and then build tests to certify that infrastructure works as designed. With this baseline, you can then create an infrastructure from nothing.

Topics

In this material, we will build web-database services in two change configuration systems: Chef and Ansible, and then use InSpec to verify that these were created as desired. We will turn three categories of tests:

• Contract Tests (Promise) - can the service respond as expected
• Conformance Test - a platform should meet the needs to host the application, e.g. apache modules required, mysql databases created, etc.
• Security Tests - follow best practices to secure the application

Slide Deck

• https://slides.com/devopsstudio/lisa18_tdi

Implementation

Tools Used:

• InSpec - infrastructure compliance test tool
• Test Kitchen - test harness to converge and verify results
• Chef - change configuration management that uses agents apply desired state
• Ansible - remote execution tool that does change configuration, infrastructure as code, and device configuration.

Guest Test Environments

These scripts require Linux guests to run, and so you need either virtualization solution or Docker. Test Kitchen by default uses Vagrant to manage virtual machines, such as Virualbox or Hyper-V.

I recommend using Docker as it is faster than running full virtual machines, and more suitable for CI solution like Jenkins.

Note that in this scenario, Docker is used as virtual virtual machines, not the common application usage, where one process stands up the container. Care has been taking to make sure the change configuration scripts and compliance test scripts detect and adjust for full virtual machine or a restricted container environment.

Host Worksation Setup

The shortest path to get minimal needed is Docker and ChefDK. Once ChefDK is installed, you will need to install kitchen-ansible plug-in for Ansible support: chef gem install kitchen-ansible.

Alternatively, I have created a small Vagrantfile, which can work to bring up a workstation suitable for running these tests.

Virtual Host Workstation Instructions

The host system will have to have Virualbox installed or Hyper-V.

vagrant up
vagrant ssh
cd lisa18_test_driven_infra

Note that this can take some time to download the Ubuntu 16.04 image initially.

Installing Directly on Mac OS X

If you have HomeBrew installed, you can get the components using:

brew cask install docker
brew tap chef/chef
brew cask install chefdk

# Local ChefDK Ruby Gems
chef gem install kitchen-ansible
chef gem install kitchen-docker

# Get the project
cd
git clone https://github.com/darkn3rd/lisa18_test_driven_infra
cd lisa18_test_driven_infra

Installing Directly on Windows

The Chef parts of this If you have Chocolatey, you can get the components running:

choco install docker-for-windows
choco install chefdk
refreshenv

# ChefDK Ruby Gems
chef gem install kitchen-ansible
chef gem install kitchen-docker

# Get the Project
cd $home
git clone https://github.com/darkn3rd/lisa18_test_driven_infra
cd lisa18_test_driven_infra

Installing Directly on Linux

Both Debian and RHEL family of distros will work. Other Linux solutions can work, but the process is more complex.

This is how you can get this installed on Ubuntu 16.04.

##### Add Docker Repo
DOCKER_REPO="https://download.docker.com/linux/ubuntu"
curl -fsSL ${DOCKER_REPO}/gpg | sudo apt-key add -
sudo add-apt-repository \
  "deb [arch=amd64] ${DOCKER_REPO} \
  $(lsb_release -cs) \
  stable"
sudo apt-get update -qq

##### Prerequisites (just in case)
sudo apt-get install -y \
    apt-transport-https \
    ca-certificates \
    software-properties-common

##### Install Docker
sudo apt-get install -y docker-ce
sudo usermod -aG docker $USER

##### ChefDK
VER=3.2.30
PKG=chefdk_${VER}-1_amd64.deb
PREFIX=https://packages.chef.io/files/stable/chefdk/${VER}
wget --quiet ${PREFIX}/ubuntu/16.04/${PKG}
sudo dpkg -i ${PKG}

# Local ChefDK Ruby Gems
chef gem install kitchen-ansible
chef gem install kitchen-docker

# Get the project
cd
git clone https://github.com/darkn3rd/lisa18_test_driven_infra
cd lisa18_test_driven_infra

Running the Tools

Once you you have the requirements installed, you can navigate to the target directory, and run the tools.

These instructions assume you are running it on the host and from the current directory of this repository. If you are running these from the virtual workstation, then you first need to run cd /vagrant.

Note: These are instructions are for POSIX shell.

Testing Apache2 configured by Chef

pushd chef/cookbooks/ez_apache
export KITCHEN_YAML=kitchen.docker.yml
# Create test environment
kitchen create
# Fix SSH private key (just in case)
chmod 600 .kitchen/docker_id_rsa
# Configure and Test
kitchen converge 
kitchen verify
# Fix Scripts
vi recipes/default.rb
# Configure and Test
kitchen converge 
kitchen verify
# Cleanup
kitchen destroy
popd

Testing MySQL configured by Chef

pushd chef/cookbooks/ez_mysql
export KITCHEN_YAML=kitchen.docker.yml
# Create test environment
kitchen create
# Fix SSH private key (just in case)
chmod 600 .kitchen/docker_id_rsa
# Configure and Test
kitchen converge 
kitchen verify
# Fix Scripts
vi recipes/default.rb
# Configure and Test
kitchen converge 
kitchen verify
# Cleanup
kitchen destroy
popd

Testing Apache2 configured by Ansible

pushd ansible/roles/ez_apache
export KITCHEN_YAML=kitchen.docker.yml
# Create test environment
kitchen create
# Fix SSH private key (just in case)
chmod 600 .kitchen/docker_id_rsa
# Configure and Test
kitchen converge 
kitchen verify
# Fix Scripts
vi tasks/main.yml
# Configure and Test
kitchen converge 
kitchen verify
# Cleanup
kitchen destroy
popd

Testing MySQL configured by Ansible

pushd ansible/roles/ez_mysql
export KITCHEN_YAML=kitchen.docker.yml
# Create test environment
kitchen create
# Fix SSH private key (just in case)
chmod 600 .kitchen/docker_id_rsa
# Configure and Test
kitchen converge 
kitchen verify
# Fix Scripts
vi tasks/main.yml
# Configure and Test
kitchen converge 
kitchen verify
# Cleanup
kitchen destroy
popd