Skip to content


Subversion checkout URL

You can clone with
Download ZIP
100644 177 lines (126 sloc) 10.8 KB
6b33222 twemcache v2.4.0 release
Manju Rajashekhar authored
1 # Twemcache: Twitter Memcached [![Build Status](](
3 Twemcache (pronounced "two-em-cache") is the Twitter Memcached. Twemcache is based on a fork of [Memcached]( v.1.4.4 that has been heavily modified to make to suitable for the large scale production environment at Twitter.
5 ## Build
7 To build twemcache from distribution tarball:
9 $ ./configure
10 $ make
11 $ sudo make install
13 To build twemcache from distribution tarball with a non-standard path to [libevent]( install:
15 $ ./configure --with-libevent=<path>
16 $ make
17 $ sudo make install
19 To build twemcache from distribution tarball with a statically linked libevent:
21 $ ./configure --enable-static=libevent
22 $ make
23 $ sudo make install
25 To build twemcache from distribution tarball in _debug mode_ with _assertion panics enabled_:
27 $ CFLAGS="-ggdb3 -O0" ./configure --enable-debug=full
28 $ make
29 $ sudo make install
31 To build twemcache from source with _debug logs enabled_ and _assertions disabled_:
33 $ git clone
34 $ cd twemcache
35 $ autoreconf -fvi
36 $ ./configure --enable-debug=log
37 $ make V=1
38 $ src/twemcache -h
40 ## Help
42 Usage: twemcache [-?hVCELdkrDS] [-o output file] [-v verbosity level]
43 [-A stats aggr interval]
44 [-t threads] [-P pid file] [-u user]
45 [-x command logging entry] [-X command logging file]
46 [-R max requests] [-c max conns] [-b backlog] [-p port] [-U udp port]
47 [-l interface] [-s unix path] [-a access mask] [-M eviction strategy]
48 [-f factor] [-m max memory] [-n min item chunk size] [-I slab size]
49 [-z slab profile]
51 Options:
52 -h, --help : this help
53 -V, --version : show version and exit
54 -E, --prealloc : preallocate memory for all slabs
55 -L, --use-large-pages : use large pages if available
56 -k, --lock-pages : lock all pages and preallocate slab memory
57 -d, --daemonize : run as a daemon
58 -r, --maximize-core-limit : maximize core file limit
59 -C, --disable-cas : disable use of cas
60 -D, --describe-stats : print stats description and exit
61 -S, --show-sizes : print slab and item struct sizes and exit
62 -o, --output=S : set the logging file (default: stderr)
63 -v, --verbosity=N : set the logging level (default: 5, min: 0, max: 11)
64 -A, --stats-aggr-interval=N : set the stats aggregation interval in usec (default: 100000 usec)
65 -t, --threads=N : set number of threads to use (default: 4)
66 -P, --pidfile=S : set the pid file (default: off)
67 -u, --user=S : set user identity when run as root (default: off)
68 -x, --klog-entry=N : set the command logging entry number per thread (default: 512)
69 -X, --klog-file=S : set the command logging file (default: off)
70 -R, --max-requests=N : set the maximum number of requests per event (default: 20)
71 -c, --max-conns=N : set the maximum simultaneous connections (default: 1024)
72 -b, --backlog=N : set the backlog queue limit (default 1024)
73 -p, --port=N : set the tcp port to listen on (default: 11211)
74 -U, --udp-port=N : set the udp port to listen on (default: 11211)
75 -l, --interface=S : set the interface to listen on (default: all)
76 -s, --unix-path=S : set the unix socket path to listen on (default: off)
77 -a, --access-mask=O : set the access mask for unix socket in octal (default: 0700)
78 -M, --eviction-strategy=N : set the eviction strategy on OOM (default: 2, random)
79 -f, --factor=D : set the growth factor of slab item sizes (default: 1.25)
80 -m, --max-memory=N : set the maximum memory to use for all items in MB (default: 64 MB)
81 -n, --min-item-chunk-size=N : set the minimum item chunk size in bytes (default: 72 bytes)
82 -I, --slab-size=N : set slab size in bytes (default: 1048576 bytes)
83 -z, --slab-profile=S : set the profile of slab item chunk sizes (default: off)
85 ## Features
87 * Supports the complete memcached ASCII protocol.
88 * Supports tcp, udp and unix domain sockets.
89 * Observability through lock-less stats collection and klogger.
90 * Pluggable eviction strategies.
91 * Easy debuggability through assertions and logging.
93 ## Slabs and Items
95 Memory in twemcache is organized into fixed sized slabs whose size is configured using the -I or --slab-size=N command-line argument. Every slab is carved into a collection of contiguous, equal size items. All slabs that are carved into items of a given size belong to a given slabclass. The number of slabclasses and the size of items they serve can be configured either from a geometric sequence with the inital item size set using -n or --min-item-chunk-size=N argument and growth ratio set using -f or --factor=D argument, or from a profile string set using -z or --slab-profile=S argument.
97 ## Eviction
99 Eviction is triggered when a cache reaches full memory capacity. This happens when all cached items are unexpired and there is no space available to store newer items. Twemcache supports the following eviction strategies, configured using the -M or --eviction-strategy=N command-line argument:
101 * No eviction (0) - don't evict, respond with server error reply.
51456e1 @thinkingfish updating to include the new eviction strategy
thinkingfish authored
102 * Item LRU eviction (1) - evict only existing items in the same slab class, least recently updated first; essentially a per-slabclass LRU eviction.
6b33222 twemcache v2.4.0 release
Manju Rajashekhar authored
103 * Random eviction (2) - evict all items from a randomly chosen slab.
3322624 @thinkingfish updating ChangeLog and
thinkingfish authored
104 * Slab LRA eviction (4) - choose the least recently accessed slab, and evict all items from it to reuse the slab.
105 * Slab LRC eviction (8) - choose the least recently created slab, and evict all items from it to reuse the slab. Eviction ignores freeq & lruq to make sure the eviction follows the timestamp closely. Recommended if cache is updated on the write path.
51456e1 @thinkingfish updating to include the new eviction strategy
thinkingfish authored
82c5d80 @thinkingfish twemcache 2.6.2 release
thinkingfish authored
107 Eviction strategies can be *stacked*, in the order of higher to lower bit. For example, `-M 5` means that if slab LRA eviciton fails, Twemcache will try item LRU eviction.
6b33222 twemcache v2.4.0 release
Manju Rajashekhar authored
109 ## Observability
111 ### Stats
113 Stats are the primary form of observability in twemcache. Stats collection in twemcache is lock-less in a sense that each worker thread only updates its thread-local metrics, and a background aggregator thread collects metrics from all threads periodically, holding only one thread-local lock at a time. Once aggregated, stats polling comes for free. There is a slight trade-off between how up-to-date stats are and how much burden stats collection puts on the system, which can be controlled by the aggregation interval -A or --stats-aggr-interval=N command-line argument. By default, the aggregation interval is set to 100 msec. You can set the aggregation interval at run time using `config aggregate <num>\r\n` command. Stats collection can be disabled at run time by passing a negative aggregation interval or at build time through the --disable-stats configure option.
115 Metrics exposed by twemcache are of three types - timestamp, counter and gauge and are collected both at the global level and per slab level. You can read about the description of all stats exposed by twemcache using the -D or --describe-stats command-line argument.
117 The following commands can be used to query stats from a running twemcache
118 * `stats\r\n`
119 * `stats settings\r\n`
120 * `stats slabs\r\n`
121 * `stats sizes\r\n`
122 * `stats cachedump <id> <limit>\r\n`
124 ### Klogger (Command Logger)
126 Command logger allows users to capture the details of every incoming request. Each line of the command log gives precise information on the client, the time when a request was received, the command header including the command, key, flags and data length, a return code, and reply message length. Few example klog lines look as follows:
128 - [09/Jul/2012:18:15:45 -0700] "set foo 0 0 3" 1 6
129 - [09/Jul/2012:18:15:46 -0700] "get foo" 0 14
130 - [09/Jul/2012:18:15:57 -0700] "incr num 1" 3 9
131 - [09/Jul/2012:18:16:05 -0700] "set num 0 0 1" 1 6
132 - [09/Jul/2012:18:16:09 -0700] "incr num 1" 0 1
133 - [09/Jul/2012:18:16:13 -0700] "get num" 0 12
135 The command logger supports lockless read/write into ring buffers, whose size can be configured with -x or --klog-entry=N command-line argument. Each worker thread logs to a thread-local buffer as they process incoming queries, and a background thread asynchronously dumps buffer contents to a file configured with -X or --klog-file=S command-line argument.
137 Since this feature has the capability of generating hundreds of MBs of data per minute, the use must be planned carefully. An enabled klog moduled can be started or stopped by sending `config klog run start\r\n` and `config klog run stop\r\n` respectively. To control the speed of log generation, the command logger also supports sampling. Sample rate can be set over with `config klog sampling <num>\r\n` command, which samples one of num commands.
139 ### Logging
141 Logging in twemcache is only available when it is built with logging enabled (--enable-debug=[full|yes|log]). By default logs are written to stderr. Twemcache can also be configured to write logs to a specific file through the -o or --output=S command-line argument.
143 On a running twemcache, we can turn log levels up and down by sending it SIGTTIN and SIGTTOU signals respectively and reopen log files by sending it SIGHUP signal. Logging levels can be set to a specific value using the `verbosity <num>\r\n` command.
145 ## Issues and Support
147 Have a bug? Please create an issue here on GitHub!
151 ## Versioning
153 For transparency and insight into our release cycle, releases are be numbered with the [semantic versioning]( format: `<major>.<minor>.<patch>` and constructed with the following guidelines:
155 * Breaking backwards compatibility bumps the major
156 * New additions without breaking backwards compatibility bumps the minor
157 * Bug fixes and misc changes bump the patch
159 ## Other Work
161 * [twemproxy]( - a fast, light-weight proxy for memcached.
162 * [twemperf]( - a tool for measuring memcached server performance.
163 * [twctop.rb]( - a tool like top for monitoring a cluster of twemcache servers.
165 ## Contributors
167 * Manju Rajashekhar ([@manju](
168 * Yao Yue ([@thinkingfish](
170 ## License
172 Copyright 2003, Danga Interactive, Inc.
174 Copyright 2012 Twitter, Inc.
176 Licensed under the New BSD License, see the LICENSE file.
Something went wrong with that request. Please try again.