Permalink
Browse files

Merge pull request #76 from matschaffer/documentation-links

Use github's relative links
  • Loading branch information...
2 parents 6126556 + 2d13f28 commit 0cc496fc20f2d56b55def0a56a137d74e05037c8 Manju Rajashekhar committed Feb 26, 2013
Showing with 19 additions and 19 deletions.
  1. +10 −10 README.md
  2. +2 −2 notes/recommendation.md
  3. +7 −7 notes/redis.md
View
@@ -35,7 +35,7 @@ To build nutcracker from source with _debug logs enabled_ and _assertions disabl
+ Supports proxying to multiple servers.
+ Supports multiple server pools simultaneously.
+ Shard data automatically across multiple servers.
-+ Implements the complete [memcached ascii](https://github.com/twitter/twemproxy/blob/master/notes/memcache.txt) and [redis](https://github.com/twitter/twemproxy/blob/master/notes/redis.md) protocol.
++ Implements the complete [memcached ascii](notes/memcache.txt) and [redis](notes/redis.md) protocol.
+ Easy configuration of server pools through a YAML file.
+ Supports multiple hashing modes including consistent hashing and distribution.
+ Can be configured to disable nodes on failures.
@@ -85,7 +85,7 @@ nutcracker can be configured through a YAML file specified by the -c or --conf-f
+ hsieh
+ murmur
+ jenkins
-+ **hash_tag**: A two character string that specifies the part of the key used for hashing. Eg "{}" or "$$". [Hash tag](https://github.com/twitter/twemproxy/blob/master/notes/recommendation.md#hash-tags) enable mapping different keys to the same server as long as the part of the key within the tag is the same.
++ **hash_tag**: A two character string that specifies the part of the key used for hashing. Eg "{}" or "$$". [Hash tag](notes/recommendation.md#hash-tags) enable mapping different keys to the same server as long as the part of the key within the tag is the same.
+ **distribution**: The key distribution mode. Possible values are:
+ ketama
+ modula
@@ -101,7 +101,7 @@ nutcracker can be configured through a YAML file specified by the -c or --conf-f
+ **servers**: A list of server address, port and weight (name:port:weight or ip:port:weight) for this server pool.
-For example, the configuration file in [conf/nutcracker.yml](https://github.com/twitter/twemproxy/blob/master/conf/nutcracker.yml), also shown below, configures 5 server pools with names - _alpha_, _beta_, _gamma_, _delta_ and omega. Clients that intend to send requests to one of the 10 servers in pool delta connect to port 22124 on 127.0.0.1. Clients that intend to send request to one of 2 servers in pool omega connect to unix path /tmp/gamma. Requests sent to pool alpha and omega have no timeout and might require timeout functionality to be implemented on the client side. On the other hand, requests sent to pool beta, gamma and delta timeout after 400 msec, 400 msec and 100 msec respectively when no response is received from the server. Of the 5 server pools, only pools alpha, gamma and delta are configured to use server ejection and hence are resilient to server failures. All the 5 server pools use ketama consistent hashing for key distribution with the key hasher for pools alpha, beta, gamma and delta set to fnv1a_64 while that for pool omega set to hsieh. Also only pool beta uses [nodes names](https://github.com/twitter/twemproxy/blob/master/notes/recommendation.md#node-names-for-consistent-hashing) for consistent hashing, while pool alpha, gamma, delta and omega use 'host:port:weight' for consistent hashing. Finally, only pool alpha and beta can speak redis protocol, while pool gamma, deta and omega speak memcached protocol.
+For example, the configuration file in [conf/nutcracker.yml](conf/nutcracker.yml), also shown below, configures 5 server pools with names - _alpha_, _beta_, _gamma_, _delta_ and omega. Clients that intend to send requests to one of the 10 servers in pool delta connect to port 22124 on 127.0.0.1. Clients that intend to send request to one of 2 servers in pool omega connect to unix path /tmp/gamma. Requests sent to pool alpha and omega have no timeout and might require timeout functionality to be implemented on the client side. On the other hand, requests sent to pool beta, gamma and delta timeout after 400 msec, 400 msec and 100 msec respectively when no response is received from the server. Of the 5 server pools, only pools alpha, gamma and delta are configured to use server ejection and hence are resilient to server failures. All the 5 server pools use ketama consistent hashing for key distribution with the key hasher for pools alpha, beta, gamma and delta set to fnv1a_64 while that for pool omega set to hsieh. Also only pool beta uses [nodes names](notes/recommendation.md#node-names-for-consistent-hashing) for consistent hashing, while pool alpha, gamma, delta and omega use 'host:port:weight' for consistent hashing. Finally, only pool alpha and beta can speak redis protocol, while pool gamma, deta and omega speak memcached protocol.
alpha:
listen: 127.0.0.1:22121
@@ -113,7 +113,7 @@ For example, the configuration file in [conf/nutcracker.yml](https://github.com/
server_failure_limit: 1
servers:
- 127.0.0.1:6379:1
-
+
beta:
listen: 127.0.0.1:22122
hash: fnv1a_64
@@ -127,7 +127,7 @@ For example, the configuration file in [conf/nutcracker.yml](https://github.com/
- 127.0.0.1:6381:1 server2
- 127.0.0.1:6382:1 server3
- 127.0.0.1:6383:1 server4
-
+
gamma:
listen: 127.0.0.1:22123
hash: fnv1a_64
@@ -141,7 +141,7 @@ For example, the configuration file in [conf/nutcracker.yml](https://github.com/
servers:
- 127.0.0.1:11212:1
- 127.0.0.1:11213:1
-
+
delta:
listen: 127.0.0.1:22124
hash: fnv1a_64
@@ -161,7 +161,7 @@ For example, the configuration file in [conf/nutcracker.yml](https://github.com/
- 127.0.0.1:11221:1
- 127.0.0.1:11222:1
- 127.0.0.1:11223:1
-
+
omega:
listen: /tmp/gamma
hash: hsieh
@@ -170,7 +170,7 @@ For example, the configuration file in [conf/nutcracker.yml](https://github.com/
servers:
- 127.0.0.1:11214:100000
- 127.0.0.1:11215:1
-
+
Finally, to make writing syntactically correct configuration file easier, nutcracker provides a command-line argument -t or --test-conf that can be used to test the YAML configuration file for any syntax error.
## Observability
@@ -208,15 +208,15 @@ Logging in nutcracker is only available when nutcracker is built with logging en
## Pipelining
-Nutcracker enables proxying multiple client connections onto one or few server connections. This architectural setup makes it ideal for pipelining requests and responses and hence saving on the round trip time.
+Nutcracker enables proxying multiple client connections onto one or few server connections. This architectural setup makes it ideal for pipelining requests and responses and hence saving on the round trip time.
For example, if nutcracker is proxing three client connections onto a single server and we get requests - 'get key\r\n', 'set key 0 0 3\r\nval\r\n' and 'delete key\r\n' on these three connections respectively, nutcracker would try to batch these requests and send them as a single message onto the server connection as 'get key\r\nset key 0 0 3\r\nval\r\ndelete key\r\n'.
Pipelining is the reason why nutcracker ends up doing better in terms of throughput even though it introduces an extra hop between the client and server.
## Deployment
-If you are deploying nutcracker in production, you might consider reading through the [recommendation document](https://github.com/twitter/twemproxy/blob/master/notes/recommendation.md) to understand the parameters you could tune in nutcracker to run it efficiently in the production environment.
+If you are deploying nutcracker in production, you might consider reading through the [recommendation document](notes/recommendation.md) to understand the parameters you could tune in nutcracker to run it efficiently in the production environment.
## Users
+ [Pinterest](http://pinterest.com/)
View
@@ -54,7 +54,7 @@ By default, nutcracker waits indefinitely for any request sent to the server. Ho
## Error Response
-Whenever a request encounters failure on a server we usually send to the client a response with the general form - `SERVER_ERROR <errno description>\r\n` (memcached) or `-ERR <errno description>` (redis).
+Whenever a request encounters failure on a server we usually send to the client a response with the general form - `SERVER_ERROR <errno description>\r\n` (memcached) or `-ERR <errno description>` (redis).
For example, when a memcache server is down, this error response is usually:
@@ -74,7 +74,7 @@ All memory for incoming requests and outgoing responses is allocated in mbuf. Mb
If nutcracker is meant to handle a large number of concurrent client connections, you should set the mbuf size to 512 or 1K bytes.
## Maximum Key Length
-The memcache ascii protocol [specification](https://github.com/twitter/twemproxy/blob/master/notes/memcache.txt) limits the maximum length of the key to 250 characters. The key should not include whitespace, or '\r' or '\n' character. For redis, we have no such limitation. However, nutcracker requires the key to be stored in a contiguous memory region. Since all requests and responses in nutcracker are stored in mbuf, the maximum length of the redis key is limited by the size of the maximum available space for data in mbuf (mbuf_data_size()). This means that if you want your redis instances to handle large keys, you might want to choose large mbuf size set using -m or --mbuf-size=N command-line argument.
+The memcache ascii protocol [specification](notes/memcache.txt) limits the maximum length of the key to 250 characters. The key should not include whitespace, or '\r' or '\n' character. For redis, we have no such limitation. However, nutcracker requires the key to be stored in a contiguous memory region. Since all requests and responses in nutcracker are stored in mbuf, the maximum length of the redis key is limited by the size of the maximum available space for data in mbuf (mbuf_data_size()). This means that if you want your redis instances to handle large keys, you might want to choose large mbuf size set using -m or --mbuf-size=N command-line argument.
## Node Names for Consistent Hashing
View
@@ -168,7 +168,7 @@
| RPUSHX | Yes | RPUSHX key value |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
-* RPOPLPUSH support requires that source and destination keys hash to the same server. You can ensure this by using the same [hashtag](https://github.com/twitter/twemproxy/blob/master/notes/recommendation.md#hash-tags) for source and destination key. Twemproxy does no checking on its end to verify that source and destination key hash to the same server, and the RPOPLPUSH command is forwarded to the server that the source key hashes to
+* RPOPLPUSH support requires that source and destination keys hash to the same server. You can ensure this by using the same [hashtag](notes/recommendation.md#hash-tags) for source and destination key. Twemproxy does no checking on its end to verify that source and destination key hash to the same server, and the RPOPLPUSH command is forwarded to the server that the source key hashes to
### Sets
@@ -204,7 +204,7 @@
| SUNIONSTORE | Yes* | SUNIONSTORE destination key [key ...] |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
-* SIDFF, SDIFFSTORE, SINTER, SINTERSTORE, SMOVE, SUNION and SUNIONSTORE support requires that the supplied keys hash to the same server. You can ensure this by using the same [hashtag](https://github.com/twitter/twemproxy/blob/master/notes/recommendation.md#hash-tags) for all keys in the command. Twemproxy does no checking on its end to verify that all the keys hash to the same server, and the given command is forwarded to the server that the first key hashes to.
+* SIDFF, SDIFFSTORE, SINTER, SINTERSTORE, SMOVE, SUNION and SUNIONSTORE support requires that the supplied keys hash to the same server. You can ensure this by using the same [hashtag](notes/recommendation.md#hash-tags) for all keys in the command. Twemproxy does no checking on its end to verify that all the keys hash to the same server, and the given command is forwarded to the server that the first key hashes to.
### Sorted Sets
@@ -245,7 +245,7 @@
| ZUNIONSTORE | Yes* | ZUNIONSTORE destination numkeys key [key ...] [WEIGHTS weight [weight ...]] [AGGREGATE SUM|MIN|MAX] |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
-* ZINTERSTORE and ZUNIONSTORE support requires that the supplied keys hash to the same server. You can ensure this by using the same [hashtag](https://github.com/twitter/twemproxy/blob/master/notes/recommendation.md#hash-tags) for all keys in the command. Twemproxy does no checking on its end to verify that all the keys hash to the same server, and the given command is forwarded to the server that the first key hashes to.
+* ZINTERSTORE and ZUNIONSTORE support requires that the supplied keys hash to the same server. You can ensure this by using the same [hashtag](notes/recommendation.md#hash-tags) for all keys in the command. Twemproxy does no checking on its end to verify that all the keys hash to the same server, and the given command is forwarded to the server that the first key hashes to.
### Pub/Sub
@@ -297,8 +297,8 @@
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
| SCRIPT LOAD | No | SCRIPT LOAD script |
+-------------------+------------+---------------------------------------------------------------------------------------------------------------------+
-
- * EVAL and EVALSHA support is limited to scripts that take at least 1 key. If multiple keys are used, all keys must hash to the same server. You can ensure this by using the same [hashtag](https://github.com/twitter/twemproxy/blob/master/notes/recommendation.md#hash-tags) for all keys. If you use more than 1 key, the proxy does no checking to verify that all keys hash to the same server, and the entire command is forwarded to the server that the first key hashes to
+
+ * EVAL and EVALSHA support is limited to scripts that take at least 1 key. If multiple keys are used, all keys must hash to the same server. You can ensure this by using the same [hashtag](notes/recommendation.md#hash-tags) for all keys. If you use more than 1 key, the proxy does no checking to verify that all keys hash to the same server, and the entire command is forwarded to the server that the first key hashes to
### Connection
@@ -383,7 +383,7 @@
### redis-benchmark against redis-server
- $ redis-benchmark -h <machine-A> -q -t set,get,incr,lpush,lpop,sadd,spop,lpush,lrange -c 100 -p 6379
+ $ redis-benchmark -h <machine-A> -q -t set,get,incr,lpush,lpop,sadd,spop,lpush,lrange -c 100 -p 6379
SET: 89285.71 requests per second
GET: 92592.59 requests per second
INCR: 89285.71 requests per second
@@ -412,4 +412,4 @@
LRANGE_300 (first 300 elements): 12376.24 requests per second
LRANGE_500 (first 450 elements): 8605.85 requests per second
LRANGE_600 (first 600 elements): 6587.62 requests per second
-
+

0 comments on commit 0cc496f

Please sign in to comment.