Skip to content
Find file
Fetching contributors…
Cannot retrieve contributors at this time
265 lines (198 sloc) 7.69 KB
"a bundle of redis threads"
Utilities maintaining and querying a replicated
partitioned (shard) Redis cluster
Formerly named 'Roma'
* Quick Start
* Poetic License
* Introduction
* Note on Partitions (Sharding)
* Configuration
o twine.conf
o redis.conf
o monitrc
* Core utilities
o twine-install
o twine-setup
o twine-start
o twine-stop
* Monit control
* Append only file
* Twine Name
Quick Start
$ git clone git://
$ cd twine
$ bash
$ bash localhost
$ bash
$ monit -c partitions/localhost/monit/monitrc
$ monit -c partitions/localhost/monit/monitrc stop all
$ bash
$ monit -c partitions/localhost/monit/monitrc quit
Poetic License
(c) 2010 Alexander E Genaud
This work ‘as-is’ we provide.
No warranty express or implied.
We’ve done our best,
to debug and test.
Liability for damages denied.
Permission is granted hereby,
to copy, share, and modify.
Use as is fit,
free or for profit.
On this notice, these rights rely.
Twine is a set of utilities that should help you get a partitioned
redis cluster up and running quickly. It handles failover using monit.
Twine does not yet promote slaves to masters. However, with enough
partitions, it is often faster and infinitely easier to bring failed
masters back online than to reconfigure slaves, masters, and clients.
Monit can be configured to monitor the cluster every second. Again,
with enough partitions, each node can be very small and thus come back
up to speed in seconds or less.
A backup is simply a matter of copying (or zipping) the partition
directory on each host. And restoration is the reverse.
Note on Partitions (Sharding)
Querying a partitioned cluster is the responsibility of the client
(perhaps I can share some Java code shortly). Basically, a client
should use the same twine.conf with the client and redis cluster.
After all, it defines the various redis nodes and their partition
number. For every key-value query, the client must first hash the
key and apply a modulo to obtain the partition to query.
For example, we could set and get from 64 partitions (shards), each
in its own redis data node on ports 4200 to 4263 on localhost:
redis-cli -p (hash(key) % 64) set key value
redis-cli -p (hash(key) % 64) get key
Twine consists of three configuration file types. There is a single
twine.conf file which specifies the redis nodes, partitions, and
network location. Each redis node has its own redis.conf, though these
are quite similar to each other. Finally, each host will have its
own monitrc file.
a simple config file which consists of redis node names, types
(data or status), partition number, host, port, and database
number. For example, the last line of a 64 part cluster might
look like this:
b4263=data 63 4263 0
This is standard redis configuration file minus all the
comments. By default, the redis data nodes of a twine cluster
will write data asynchronously at least once per minute. All
of this can be changed either in the various redis.conf or in
twine-setup that generates them.
monitrc is used by monit - a unix service monitor. One monitrc
is generated by twine-setup for each host. The first line tells
monit how often it should check the cluster. By default, it is
set to five seconds. Httpd is required for some basic queries
like status and summary.
Core utilities
twine (not yet available)
wraps redis-cli, but hashes the key
and executes the command on the
appropriate partition (shard)
twined (not yet available)
daemon (of which there may be many)
which monitors the cluster. It
competes with any other twined for
alpha position. The alpha sets the
cluster status and handles fail-over.
It can be configured to send an email
if the status is HALT or any Redis
node is down for more than x seconds.
twine-setup [host]
creates the entire cluster plumbing by default
including all partitions and redis.conf
according to the twine.conf. Optionally takes
a host name as arguement. This is convenient for
setting up each machine using the same twine.conf
file. The host name must be listed in the twine.conf
twine-start [host]
starts the entire cluster according
to the twine.conf configuration file
(must be run separately per machine)
twine-stop [host]
stops the entire cluster according
to the twine.conf configuration file
twine-status (not yet available)
returns the cluster status in various
formats: summary, list, xml, html
twine-hash (not yet available)
A plug-able hash function wrapper
which accepts two arguments:
a single 7-bit ASCII string (key)
and a modulus (number of partitions).
It returns the partition number.
twine-backup (not yet available)
backs up the entire cluster to
directory(ies) according to twine.conf
twine-restore (not yet available)
overwrites all cluster data from a
previous backup.
Monit control
Twine-setup configures a monitrc control file for
each host, found in partitions/HOSTNAME/monit.
It is worth editing the monitrc file (or twine-setup)
to configure how often (in seconds) monit checks the status
of the cluster (first line set daemon x). Also, mail
alerts are worth configuring, or removing so as not to
clutter the log files.
To start monit:
monit -c partitions/HOSTNAME/monit/monitrc
Try to kill some or all of the redis nodes (not in production, of
course) with the following commands and see how quickly monit
brings them back to life:
kill -9 <PID>
If find the following alias helpful, which displays monit and
redis processes:
alias psef='ps -ef | grep -v volume-monitor | grep "[rm][eo][dn]i[st]"'
When the monitrc file is to your liking, you will probably want to
move it to '/etc/monit/monitrc' and/or run it automatically on startup,
from init.d.
Append only file
Twine-setup contains a variable at the top APPENDONLY=yes|no
which sets each redis not in the cluster to save using
either the original asynchronous database dump (no) or the
new append only journalling (yes).
Setting APPENDONLY to 'yes' will create a script in
partitions/HOSTNAME/appendonly/ which can be
invoked by crontab. This script will compact the journal
file asynchronously.
Twine Name
Why Twine? Twine is a strong chord consisting of multiple threads.
Twine is used to bind things together. Likewise Twine, the utility,
helps bind a cluster of Redis threads.
What about Roma? Well, within two hours of creating the public github
repo, I was informed by two people that there was a distributed key-
value store named Roma and the names might confuse people. I agreed.
Something went wrong with that request. Please try again.