memcached development tree
C Perl M4 DTrace C++ Shell Other
Latest commit ae84d77 Feb 23, 2017 @dormando dormando refactor chunk chaining for memory efficiency
Memory chunk chains would simply stitch multiple chunks of the highest slab
class together. If your item was 17k and the chunk limit is 16k, the item
would use 32k of space instead of a bit over 17k.

This refactor simplifies the slab allocation path and pulls the allocation of
chunks into the upload process. A "large" item gets a small chunk assigned as
an object header, rather than attempting to inline a slab chunk into a parent
chunk. It then gets chunks individually allocated and added into the chain
while the object uploads.

This solves a lot of issues:

1) When assembling new, potentially very large items, we don't have to sit and
spin evicting objects all at once. If there are 20 16k chunks in the tail and
we allocate a 1 meg item, the new item will evict one of those chunks
inbetween each read, rather than trying to guess how many loops to run before
giving up. Very large objects take time to read from the socket anyway.

2) Simplifies code around the initial chunk. Originally embedding data into
the top chunk and embedding data at the same time required a good amount of
fiddling. (Though this might flip back to embedding the initial chunk if I can
clean it up a bit more).

3) Pulling chunks individually means the slabber code can be flatened to not
think about chunks aside from freeing them, which culled a lot of code and
removed branches from a hot path.

4) The size of the final chunk is naturally set to the remaining about of
bytes that need to be stored, which means chunks from another slab class can
be pulled to "cap off" a large item, reducing memory overhead.
Permalink
Failed to load latest commit information.
devtools some whitespace test fixes. May 29, 2016
doc typo fix for doc/protocol.txt Feb 26, 2017
m4 Backporting AC_PROG_CC_C99 macro for redhat. Feb 20, 2010
scripts whitespace fix. Feb 26, 2017
t refactor chunk chaining for memory efficiency Mar 20, 2017
.gitignore .gitignore updates. Jul 13, 2016
.shipit Checking in changes prior to tagging of version 1.2.4. Changelog diff… Dec 6, 2007
AUTHORS autoconf/automake support from evan Jun 14, 2003
BUILD better build instructions Jul 31, 2003
CONTRIBUTING.md I think the README was 12 years old. Feb 26, 2017
COPYING add a duplicate of the LICENSE file so GNU automake doesn't assume it… Aug 6, 2004
ChangeLog Update old ChangeLog note to visit Github wiki Oct 12, 2016
Doxyfile fix typos Jan 1, 2015
HACKING Add links to http://contributing.appspot.com/memcached Mar 20, 2010
LICENSE change from GPL to BSD. Jun 15, 2003
LICENSE.bipbuffer initial logger code. Jun 17, 2016
LICENSE.itoa_ljust import itoa_ljust.c for fast number printing Jan 8, 2017
Makefile.am import itoa_ljust.c for fast number printing Jan 8, 2017
NEWS danga.com -> memcached.org in a few spots. Apr 30, 2014
README.md I think the README was 12 years old. Feb 26, 2017
assoc.c clean up global stats code a little. Jun 27, 2016
assoc.h use item lock instead of global lock when hash expanding. Dec 26, 2014
autogen.sh fix configure.ac warning and use system automake Nov 18, 2015
bipbuffer.c fix solaris compiler warning in bipbuffer.c Jun 21, 2016
bipbuffer.h initial logger code. Jun 17, 2016
cache.c allow skipping thread-local cache locks Jan 8, 2017
cache.h allow skipping thread-local cache locks Jan 8, 2017
configure.ac don't fail on systems without 64bit atomics Jul 2, 2016
crawler.c stop using atomics for item refcount management Jan 23, 2017
crawler.h pull LRU crawler out into its own file. Aug 20, 2016
daemon.c Build fixes for ubuntu 8.10/64. Jan 20, 2009
globals.c clean up global stats code a little. Jun 27, 2016
hash.c Make hash table algorithm selectable Apr 17, 2014
hash.h Make hash table algorithm selectable Apr 17, 2014
items.c refactor chunk chaining for memory efficiency Mar 20, 2017
items.h refactor chunk chaining for memory efficiency Mar 20, 2017
itoa_ljust.c conditionally don't inline the ASCII VALUE line Jan 8, 2017
itoa_ljust.h import itoa_ljust.c for fast number printing Jan 8, 2017
jenkins_hash.c fix typos Jan 1, 2015
jenkins_hash.h Make hash table algorithm selectable Apr 17, 2014
logger.c log class id with fetchers and mutations Jan 5, 2017
logger.h log class id with fetchers and mutations Jan 5, 2017
memcached.c refactor chunk chaining for memory efficiency Mar 20, 2017
memcached.h refactor chunk chaining for memory efficiency Mar 20, 2017
memcached.spec.in danga.com -> memcached.org in a few spots. Apr 30, 2014
memcached_dtrace.d Define touch command probe for DTrace support Jul 29, 2012
murmur3_hash.c fix typos Jan 1, 2015
murmur3_hash.h Make hash table algorithm selectable Apr 17, 2014
protocol_binary.h fix typos Jan 1, 2015
sasl_defs.c fix enable-sasl-pwdb Sep 18, 2012
sasl_defs.h Specify hostname in sasl_server_new. Jan 28, 2012
sizes.c Created a tool to show us the sizes of various data structures. Mar 10, 2009
slabs.c refactor chunk chaining for memory efficiency Mar 20, 2017
slabs.h refactor chunk chaining for memory efficiency Mar 20, 2017
solaris_priv.c Remove compilation errors from gcc on Solaris Aug 23, 2009
stats.c Make hash table algorithm selectable Apr 17, 2014
stats.h stats_prefix_* should not assume that all keys are zero-terminated Jan 3, 2009
testapp.c gatkq: return key in response Jan 1, 2015
thread.c use LRU thread for COLD -> WARM bumps Jan 31, 2017
timedrun.c bug237: Don't compute incorrect argc for timedrun Jan 12, 2012
trace.h Backport binary TOUCH/GAT/GATQ commands Sep 27, 2011
util.c Print with more-restricted format string to fix compiler warning Feb 26, 2017
util.h "lru tune" command for setting LRU parameters Jan 30, 2017
version.pl Make autoversioning/spec gen work better. Nov 2, 2009
version.sh Generate the memcached.spec along with the version number. Aug 29, 2009

README.md

Memcached

Memcached is a high performance multithreaded event-based key/value cache store intended to be used in a distributed system.

See: https://memcached.org/about

A fun story explaining usage: https://memcached.org/tutorial

If you're having trouble, try the wiki: https://memcached.org/wiki

If you're trying to troubleshoot odd behavior or timeouts, see: https://memcached.org/timeouts

https://memcached.org/ is a good resource in general. Please use the mailing list to ask questions, github issues aren't seen by everyone!

Dependencies

Environment

Be warned that the -k (mlockall) option to memcached might be dangerous when using a large cache. Just make sure the memcached machines don't swap. memcached does non-blocking network I/O, but not disk. (it should never go to disk, or you've lost the whole point of it)

Website

Contributing

See https://github.com/memcached/memcached/wiki/DevelopmentRepos