Reference guide for Cloudspin
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
00-starting
01-basic-stack
02-environments
03-aws-setup
04-pipelines
05-split-stacks
node_modules
reference
.gitignore
CODE_OF_CONDUCT.md
LICENSE.txt
Makefile
README.adoc
SUMMARY.adoc
book.json

README.adoc

What is Cloudspin?

Cloudspin is a collection of code I (Kief Morris) am using on to explore ways to structure and manage infrastructure as code projects. I am using it to help illustrate concepts, patterns, and practices from my book, Infrastructure as Code, articles, and workshops. I am also using it with client teams I work with at ThoughtWorks to work out ideas for project structures and tooling.

This could potentiallly evolve into a community-driven set of tools and template projects (replacing "I" with "we" in this documentation!) For the moment, I’m making fairly radical changes on an unpredictable basis, so it’s likely to be inconsistent and even broken at times.

Feel free to copy and use material from here (subject to the terms of the MIT license), but you will need to modify and extend it to make it useful for your own project. I am not currently maintaining this as a project that assumes people are using it directly, so there is no attempt to preserve backwards compatibility, provide upgrade paths. In other words, copy and use this stuff at your own risk.

Note

The Cloudspinners repositories on Github have the source code of the cloudspin tools, example projects, and this reference documentation.

Why Cloudspin?

Currently, most people and teams managing infrastructure with tools such as Terraform, CloudFormation, etc. define their own unique project structures, and write their own unique wrapper scripts (using make, shell, rake, or whatever) to run that tool and associated tasks. Essentially, each project is a special snowflake. Some tools (such as Terragrunt), are emerging to create common structures and build tooling for infrastructure.

Cloudspin uses its own structure and tooling for building projects, in order to support the concepts I’m using it to work out. (As mentioned, these are changing and evolving without any attempt at consistency or support. Their purpose is exploration, not a product.) These are based around the lifecycle of "http://infrastructure-as-code.com/patterns/2018/03/28/defining-stacks.html[stacks]" - groups of infrastructure elements provisioned on dynamic infrastructure such as IaaS clouds.

Over time, a more stable set of structures and tooling may emerge from this. The hypothesis is that, with a common project structure and tooling, teams can:

  • Spend less time creating and maintaining scaffolding, and more time on higher value work,

  • Bring new team members up to speed more quickly,

  • Rapidly create new infrastructure projects, and add new infrastructure to existing projects,

  • More easily debug and improve infrastructure,

  • Adopt mature engineering practices such as TDD and CD for their infrastructure,

  • Have more fully featured infrastructure sooner in a project.

Additionally, common conventions and structures for infrastructure projects can foster an ecosystem:

  • People can create and share tools and scripts that work with the common structure, creating an ecosystem,

  • People can then create and share infrastructure code for running various software and services, creating a community library.

Philosophy

What is in Cloudspin

At the moment, cloudspin includes:

  • The cloudspin-stack Ruby library, which provides classes for interacting with stack definitions (source code) and instances. This is currently implemented for Terraform projects, for AWS. Other cloud providers should be easy enough to implement. In theory it can be adapted to use tools other than Terraform (e.g. CloudFormation), but in practice it’ll probably need a fair bit of work.

  • Some conventions for structuring project folders for infrastructure stacks, for configuring and provisioning infrastructure stack instances, for packaging infrastructure (Terraform) code into an artefact, and using it in a delivery pipeline (currently with AWS CodePipeline).

  • A command-line tool (included in the cloudspin-stack gem), and some rake tasks which can used to manage stack instances using the cloudspin-stack classes, following the conventions described here.

  • An example stack project, spin-stack-network, using these conventions and tools.

What I’d like to add over time, with the help of a community: - Conventions and tools for configuring stack instances from different sources (files in the project, key/value stores) - Conventions and tools for integrating multiple stacks - Conventions and tools for organizing and managing collections of stacks in environments - More example projects - Develop a library of template stack projects that people can share and use - Provide examples and templates for common core services - CI and CD tools, monitoring, log aggregation, etc. - Add support and examples for more cloud providers - Consider how to support additional infrastructure tools beyond Terraform

This documentation

The purpose of this reference guide is to explain the conventions the cloudspin code are meant to support, and how to use them. It’s quite likely to lag the actual code, so fair warning.