Skip to content

An example set of Terraform modules, Lambda scripts, Packer templates, Ansible playbooks and Puppet manifests that provision and configure a set of machines in AWS to help coders get started in DevOps

License

dwp/devops-workstream

master
Switch branches/tags

Name already in use

A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch?
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
bin
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Workstream

About

This repo contains an example set of Terraform modules, Lambda scripts, Packer templates, Ansible Playbooks and Puppet manifests that provision and configure a set of machines in AWS. It's designed to help coders get started in DevOps.

Getting started

This repo is designed to be a simple starting point for infrastructure-as-code projects. Invariably, you'll need some tools to build that infrastructure and credentials to secure access to it. The setup of both is described here:

If you've used the default key name and path (described above), then to see what terraform plans to build out in the default region (eu-west-2):

terraform plan

If it seems sensible, apply it:

terraform apply

Look at what you've created in the AWS console!

Shutting down

Most important of all while developing IAC, clear it up afterwards:

terraform destroy

Also be aware that while terraform destroy will remove all the instantiated hosts/security groups/subnets/VPCs etc., it will not remove everything. You'll need to use the AWS console to manually deregister/delete:

  • Volumes belonging to hosts instantiated by Terraform
    • After multiple terraform-plan-apply-destroy cycles, you can easily accumulate a few dozen volumes. These will persist (at a cost) unless explicitly removed.
  • AMIs produced by Packer
    • Also the Snapshots that Packer produces as an intermediate file.

File structure

This repo is organised at the top-level by technology.

  • /bin - a few scripts to hold useful commands for reference
  • /docs - markdown-formatted documents describing the examples in this repo

Image creation and orchestration

  • /packer - a set of Packer templates
    • centos_updates.json - create updated AMI image based on CentOS 7 (cross-region)
    • remote_provisioning.json - create AMI image based on CentOS 7, including this repo with all pre-requisites installed

Config management

  • /ansible - an ansible control folder (/etc/ansible) containing an array of playbooks and roles
    • roles
      • common - a simple common role shared across all playbooks
  • /puppet - a Puppet control folder (/etc/puppet) containing an array of environments and modules
    • environments
      • workstream - the main workstream template environment
        • manifests - puppet manifests
          • host-<hostname>.pp - puppet masterless manifest, called directly with puppet apply
          • site.pp - general environment master manifest, server via Puppet master
        • modules - environment/DWP specific modules
    • modules - community modules

Provisioning

  • /terraform - a collection of modules to provision machines, linked from single root module main.tf.
    • ansible - a terraform module to set up a generic host and invoke ansible on it locally
    • aws_background - a terraform module to set up a basic AWS environment
    • janitor - a terraform module to start/stop instances based on their tags
    • pack_amis - a terraform module to invoke packer
    • packer - a terraform module to instantiate a host from a packed machine image
    • puppetmless - a terraform module to set up a generic host and invoke puppet apply on it locally
    • puppetmastered - a terraform module to set up a generic host, connect it to a puppetmaster and set up a puppet agent for repeat runs

Testing

To run the integration test suite in /tests/awspec, you need install the gems bundle install

The test run command is as simple as bundle exec rake spec after a successful terraform apply, but you need explicitly set the region environment variable:

AWS_DEFAULT_REGION="eu-west-2" bundle exec rake spec

Security

All hosts are built with SELinux enabled (enforcing).

Documentation

We've taken the same simple approach to documentation. It's all in markdown-formatted .md files, linked directly from this README.md.

About

An example set of Terraform modules, Lambda scripts, Packer templates, Ansible playbooks and Puppet manifests that provision and configure a set of machines in AWS to help coders get started in DevOps

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published