/example-ci-circleci

This example shows you how to use the Fugue client in CircleCI builds in order to deploy and update infrastructure.
  1. Makefile 100.0%
Makefile
Clone or download

Clone with HTTPS

Use Git or checkout with SVN using the web URL.

Open in Desktop Download ZIP
Find file
Switch branches/tags
Nothing to show
Latest commit e32b0ca Feb 14, 2017 @dom-at-fugue dom-at-fugue Create LICENSE
Permalink
Failed to load latest commit information.
images initial commit Dec 12, 2016
.gitignore initial commit Dec 12, 2016
Example.lw Comment out ssh access Dec 16, 2016
LICENSE Create LICENSE Feb 14, 2017
Makefile initial commit Dec 12, 2016
README.md Adding links Jan 18, 2017
circle.yml Fixing logic Dec 16, 2016

README.md

example-ci-circleci

Overview

About This Example

This example demonstrates continuous integration of cloud infrastructure using Fugue and CircleCI. CircleCI uses Fugue to build and update your infrastructure for you in each deployment stage of the Gitflow model simply by pushing to the correct branch in Github. The composition shown in this example creates a network in Amazon Web Services.

Prerequisites

  • Obtain an AWS IAM user with an AdministratorAccess policy applied (see "Setup" below).
  • Obtain the Fugue client tools here.
  • Note: For the purpose of this example we made the artifacts available from Artifactory. You will need to download the client and make it available to the CircleCI build workers. The urls in the circle.yml will need to be updated accordingly.
  • Obtain a CircleCI account here.

Usage

Enable the Github project in CircleCI. The CircleCI build specification is defined in circle.yml. Please refer to the CircleCI documentation for details. CircleCI builds are triggered on pushes to the Github project branches. After pushing a change to a particular branch, the build will begin executing the build steps as defined in circle.yml. The Fugue client will be installed as part of the machine stage. Fugue compile-time validations and dry-run checks will be executed during the testing phase. Assuming tests pass, the deployment stages will initiate a fugue run to launch the example composition as a process, or execute a fugue update if a process with the Fugue alias already exists. The alias is dynamically named based on the CIRCLE_BRANCH environment variable whose value is the git branch currently building, or it is given one of the hardcoded values for the develop, staging, and production environments. We use fugue status to check if an alias exists for a given deployment stage, and if it does, we issue a fugue update instead of a fugue run.

The deployment stages are modeled after the Gitflow branching model.

Environment Branch Name Fugue Alias
feature feature regex dynamic based on branch name
develop develop develop
staging release regex staging
production master production

Setup

Create an IAM user with AdministratorAccess rights.

ci

Install Fugue. Refer to the Fugue documentation for details.

Initialize the project directory. Refer to the Fugue documentation and ensure you have the correct AMI ID from the Download Portal.

$ fugue init ami-4abb8a5d
[ fugue init ] Initializing Fugue project with the following configuration:

Fugue Conductor AMI ID: ami-4abb8a5d
AWS Credentials: Environment variables

Validating Fugue Conductor AMI ID ...
[ OK ] Provided AMI ID is valid.

Creating new fugue.yaml file ...

[ Done ] Project initialized.

ci

Launch the conductor.

$ fugue install
[ fugue install ] Installing Fugue Conductor

Install Details:

   Conductor AMI ID: ami-4abb8a5d
   AWS Account: fugue-ci-demo/xxxxxxxxxxxx

[ WARN ] Would you like to proceed with installing? [y/N]: y
Installing the Fugue Conductor into AWS account fugue-ci-demo/xxxxxxxxxxxx.

FugueVpc                             Complete
FugueSubnet2RouteTableAssociation    Complete
FugueHealthCheckDb                   Complete
FugueInstanceProfile                 Complete
FugueRouteTable                      Complete
FugueLaunchConfiguration             Complete
FugueSubnet1RouteTableAssociation    Complete
FugueVpcGateway                      Complete
FugueInternetRoute                   Complete
FugueSubnet2                         Complete
FugueSubnet1                         Complete
FugueIam                             Complete
FugueAutoScalingGroup                Complete
FugueResourceEventsTopic             Complete
FugueVpcSecurityGroup                Complete
FugueVpcGatewayAttachment            Complete
-----------------------------------------------
Overall Progress  [#########################]  100%

[ HELP ] Exiting the install command while in progress (CTRL+C) will only stop progress tracking and *not* the install itself.

[ OK ] Fugue Conductor installed.

Booting the Conductor, please wait as this may take between 5-15 minutes...

[ HELP ] The Conductor needs to boot before it can accept commands from the CLI. Exiting the install command while in progress (CTRL+C) will only stop progress tracking and *not* the install itself or the booting process.

[ DONE ] Fugue has been successfully installed and is ready to receive commands.

ci

Set up CircleCI.

ci

Running The Composition

In this example, CircleCI is going to be using Fugue to run the composition for us. We'll just be committing to Github using the Gitflow branching model to promote infrastructure changes from environment to environment until the changes make their way to the production workload. We start off with a running conductor and an empty AWS account.

ci

ci

Let's start by pushing the develop branch up to Github. In a few minutes you should see the develop workload running in Fugue.

ci

A new VPC will be created in our AWS account with the security group rules we've defined in our composition.

ci

Let's make some changes to the security group rules. We'll add a rule to allow inbound OpenSSH access. Following Gitflow, we branch off develop and create a feature branch to make our changes and push them up to Github. Notice that Fugue isolates each workload in its own VPC. The Fugue alias will be named after the branch name we chose for our feature branch.

ci

The new feature/allow-inbound-openssh workload should have an additional rule to allow OpenSSH access.

ci

Now that we're happy with our changes to the security group, let's merge those changes back into develop and cut a release. We'll do this by sending a pull request from the feature branch we created to the develop branch. After code review and a few LGTM's, we can merge the pull request.

ci

Merging the changes into the develop branch will trigger a new build in CircleCI.

ci

When the build is complete, Fugue should update the running develop workload with the changes we added in the feature branch.

ci

Let's cut a release and deploy these changes to our staging environment so we can test it out and make sure everything is perfect before we promote these changes into production. By using Fugue with CircleCI, we're able to deploy our infrastructure into our AWS environments and promote changes simply by committing to git.

Here you can see CircleCI running the tests. Fugue's compile-time validations ensure that our changes will be successful and Fugue's dry-run capability shows the changes that are going to be applied to the running infrastructure, giving reviewers more confidence about merging in contributions.

ci

After the tests pass, the release will be deployed to our staging environment.

ci

The fugue status output displays our running workloads.

ci

In AWS we should have three VPCs, each with a security group that allows inbound OpenSSH access.

ci

Finally, let's merge our release branch into master. CircleCI will run our build and deploy our changes into our production environment.

ci

Verifying Results

We should now have four running workloads in AWS modeled after the Gitflow branching model.

ci

Shutting It Down

In a production environment, we'd want to keep the process aliased production running, and we would likely terminate the other three processes. Since this is a demonstration, we are going to terminate all four processes. This is as simple as executing fugue kill -y <alias> for every process, which instructs Fugue to tear down the cloud infrastructure it created for each workload.

$ fugue kill -y feature/allow-inbound-openssh
[ fugue kill ] Killing running process with Alias: feature/allow-inbound-openssh

Requesting the Conductor to kill running composition with Alias: feature/allow-inbound-openssh...
[ Done ] The conductor is killing the process with Alias: feature/allow-inbound-openssh


$ fugue kill -y develop
[ fugue kill ] Killing running process with Alias: develop

Requesting the Conductor to kill running composition with Alias: develop...
[ Done ] The conductor is killing the process with Alias: develop


$ fugue kill -y staging
[ fugue kill ] Killing running process with Alias: staging

Requesting the Conductor to kill running composition with Alias: staging...
[ Done ] The conductor is killing the process with Alias: staging


$ fugue kill -y production
[ fugue kill ] Killing running process with Alias: production

Requesting the Conductor to kill running composition with Alias: production...
[ Done ] The conductor is killing the process with Alias: production

ci

Run fugue status, and you will see that the processes have transitioned to the Killing state. If you run fugue status again after Fugue is done killing the processes, you will see a message indicating that cleanup has finished successfully.

Fugue Status Report for user/xxxxxxxxxxxx - Mon Dec 12 2016 11:13am

State    Updated    Created    FID    Alias    Last Message
-------  ---------  ---------  -----  -------  --------------
Nothing to see here. Go create something! :-)