Create/restore a riak cluster on any cloudprovider using Terraform
Shell CoffeeScript Makefile
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
clusters/example
payload
test
.gitignore
LICENSE
Makefile
README.md
control.sh
package.json

README.md

riak-formation

Create/restore a riak cluster on any cloudprovider using Terraform

What

  • Riak is a fine distributed key-value store built by Basho
  • Riak CS is an addon so that you can build your own S3 compatible storage service, with Riak as a backend.
  • Terraform is a tool to launch clusters in the cloud, using nothing but simple configuration files.

riak-formation is a collection of config files and scripts that let you create/restore a Riak (CS) cluster in the cloud using a single command.

Why

Launching a Riak cluster can be involved. Hence Basho has released a few scripts for AWS CloudFormation to automate this.

While these scripts are more advanced than what riak-formation is aiming for - they tie you in with Amazon's EC2. The Terraform approach also works with other vendors such as e.g. Digital Ocean. You could even mix multiple providers to some extent.

Terraform works idempotent, meaning if it encounters servers that diverged from our config, we can just run it again and it will make all the required changes to restore our cluster as it was defined.

Terraform config uses the HLC configuration language and is more dense than the 48.189 kb CloudFormation JSON files, so it's easy to spot mistakes, and more fun to work on.

How

First, riak-formation needs to know which cloud provider you wish to target, and the associated account. You can pass it these via cluster config. Either directly on the commandline, or add them to clusters/production/config.sh. production can be any name, you you could be deploying many completely different riak clusters from this project. Since config.sh supports many configuration options, it is not required to define your own cluster's infrastructure definitions (*.tf), it instead will automatically borrow default.tf from the example cluster to reduce duplication. SSH keys, SSL keys, config.sh, state files, and plan files, do have to be administerd on a cluster-by-cluster basis.

Now type make launch. riak-formation will launch as many machines as you defined, set up firewalls, install riak nodes, connect them together, set up Riak Control, and end with this summary:

screen shot 2014-11-21 at 21 12 26

Click any of the links, and you're right inside the control panel:

screen shot 2014-11-21 at 21 12 39

Todo

  • Make env file dictate ingress, ports, server count, etc. So that you can launch different Riak clusters by just sourcing different env files.
  • S3 emulation
  • Can we serve read-only json files
  • Multi datacenter replication? Can we do that for free somehow?
  • Enable Search
  • Research/Document CRDTs in 2.0
  • Backup / Restore
  • Simplify directory layout
  • Store assembly as MAP:

"Maps are the richest of the Riak Data Types because within the fields of a map you can nest any of the five Data Types, including maps themselves (you can even embed maps within maps, and maps within those maps, and so on). " - http://docs.basho.com/riak/latest/theory/concepts/crdts/

Credits

This project draws from