Create/restore a riak cluster on any cloudprovider using Terraform
- Riak is a fine distributed key-value store built by Basho
- Riak CS is an addon so that you can build your own S3 compatible storage service, with Riak as a backend.
- Terraform is a tool to launch clusters in the cloud, using nothing but simple configuration files.
riak-formation is a collection of config files and scripts that let you create/restore a Riak (CS) cluster in the cloud using a single command.
While these scripts are more advanced than what riak-formation is aiming for - they tie you in with Amazon's EC2. The Terraform approach also works with other vendors such as e.g. Digital Ocean. You could even mix multiple providers to some extent.
Terraform works idempotent, meaning if it encounters servers that diverged from our config, we can just run it again and it will make all the required changes to restore our cluster as it was defined.
First, riak-formation needs to know which cloud provider you wish to target, and the associated account. You can pass it these via cluster config. Either directly on the commandline, or add them to
production can be any name, you you could be deploying many completely different riak clusters from this project.
config.sh supports many configuration options, it is not required to define your own cluster's infrastructure definitions (
*.tf), it instead will automatically borrow
default.tf from the
example cluster to reduce duplication. SSH keys, SSL keys,
config.sh, state files, and plan files, do have to be administerd on a cluster-by-cluster basis.
make launch. riak-formation will launch as many machines as you defined, set up firewalls, install riak nodes, connect them together, set up Riak Control, and end with this summary:
Click any of the links, and you're right inside the control panel:
- Make env file dictate ingress, ports, server count, etc. So that you can launch different Riak clusters by just sourcing different env files.
- S3 emulation
- Can we serve read-only json files
- Multi datacenter replication? Can we do that for free somehow?
- Enable Search
- Research/Document CRDTs in 2.0
- Backup / Restore
- Simplify directory layout
- Store assembly as MAP:
"Maps are the richest of the Riak Data Types because within the fields of a map you can nest any of the five Data Types, including maps themselves (you can even embed maps within maps, and maps within those maps, and so on). " - http://docs.basho.com/riak/latest/theory/concepts/crdts/
This project draws from
- Installing on Debian and Ubuntu
- How To Create a Riak Cluster on an Ubuntu VPS
- Call Me Maybe: Carly Rae Jepsen and the Perils of Network Partitions - RICON East 2013
- Jepsen: ZK, NuoDB, Kafka, & Cassandra
- Kyle Kingsbury and Al Tobey - Cassandra and Go Doubleheader
- Readings in conflict-free replicated data types
- A comprehensive study of Convergent and Commutative Replicated Data Types