Skip to content
This repository

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

memcached development tree

fix LRU contention for first minute of uptime

Noticed in benchmarks that memcached was slow for the first minute. ~350k
gets/sec max, then it would jump up to millions/sec.

    if (it->time < current_time - ITEM_UPDATE_INTERVAL) {
current_time starts at 2, which then underflows toward 4 billion, which is
larger than it->time, so all fetches for the first minute require shuffling
the LRU. Which happens under a global lock.

This probably goofed up all short-term benchmarks, and potentially slowed down
high traffic instances which were recently restarted.

Unfortunately this means the tests no longer exercise item_update(), so some
more work is necessary.
latest commit d7324b0bd1
dormando dormando authored April 17, 2014
Octocat-spinner-32 devtools whitespace fix for freebsd July 29, 2012
Octocat-spinner-32 doc lru_crawler_tocrawl to limit inspected per run April 17, 2014
Octocat-spinner-32 m4 Backporting AC_PROG_CC_C99 macro for redhat. February 20, 2010
Octocat-spinner-32 scripts Implement settings and sizes stats. Show information about empty slabs. December 19, 2013
Octocat-spinner-32 t lru_crawler_tocrawl to limit inspected per run April 17, 2014
Octocat-spinner-32 .gitignore Generate the memcached.spec along with the version number. August 29, 2009
Octocat-spinner-32 .shipit Checking in changes prior to tagging of version 1.2.4. Changelog diff… December 06, 2007
Octocat-spinner-32 AUTHORS autoconf/automake support from evan June 14, 2003
Octocat-spinner-32 BUILD better build instructions July 31, 2003
Octocat-spinner-32 CONTRIBUTING Add links to http://contributing.appspot.com/memcached March 20, 2010
Octocat-spinner-32 COPYING add a duplicate of the LICENSE file so GNU automake doesn't assume it… August 06, 2004
Octocat-spinner-32 ChangeLog Issue 131 - ChangeLog is outdated November 10, 2010
Octocat-spinner-32 Doxyfile Exclude testapp.c instead of internal_tests.c from docs. April 17, 2009
Octocat-spinner-32 HACKING Add links to http://contributing.appspot.com/memcached March 20, 2010
Octocat-spinner-32 LICENSE change from GPL to BSD. June 15, 2003
Octocat-spinner-32 Makefile.am Make hash table algorithm selectable April 16, 2014
Octocat-spinner-32 NEWS auto* changes, update to version 1.0.2 June 14, 2003
Octocat-spinner-32 README.md Use Markdown for README. July 29, 2012
Octocat-spinner-32 assoc.c Make hash table algorithm selectable April 16, 2014
Octocat-spinner-32 assoc.h item locks now lock hash table buckets September 03, 2012
Octocat-spinner-32 autogen.sh Added support for automake 1.13 & 1.14 December 09, 2013
Octocat-spinner-32 cache.c totally destroy test caches July 29, 2012
Octocat-spinner-32 cache.h Added documentation on the cache interface April 17, 2009
Octocat-spinner-32 configure.ac Add linux accept4() support. December 19, 2013
Octocat-spinner-32 daemon.c Build fixes for ubuntu 8.10/64. January 20, 2009
Octocat-spinner-32 globals.c slab reassignment December 19, 2011
Octocat-spinner-32 hash.c Make hash table algorithm selectable April 16, 2014
Octocat-spinner-32 hash.h Make hash table algorithm selectable April 16, 2014
Octocat-spinner-32 items.c fix LRU contention for first minute of uptime April 17, 2014
Octocat-spinner-32 items.h optionally take a list of slabs to run against. April 17, 2014
Octocat-spinner-32 jenkins_hash.c Make hash table algorithm selectable April 16, 2014
Octocat-spinner-32 jenkins_hash.h Make hash table algorithm selectable April 16, 2014
Octocat-spinner-32 memcached.c fix LRU contention for first minute of uptime April 17, 2014
Octocat-spinner-32 memcached.h fix LRU contention for first minute of uptime April 17, 2014
Octocat-spinner-32 memcached.spec.in stupid whitespace thing pulls in dead files July 29, 2012
Octocat-spinner-32 memcached_dtrace.d Define touch command probe for DTrace support July 29, 2012
Octocat-spinner-32 murmur3_hash.c Make hash table algorithm selectable April 16, 2014
Octocat-spinner-32 murmur3_hash.h Make hash table algorithm selectable April 16, 2014
Octocat-spinner-32 protocol_binary.h Add binary GATK/GATKQ September 27, 2011
Octocat-spinner-32 sasl_defs.c fix enable-sasl-pwdb September 18, 2012
Octocat-spinner-32 sasl_defs.h Specify hostname in sasl_server_new. January 27, 2012
Octocat-spinner-32 sizes.c Created a tool to show us the sizes of various data structures. March 10, 2009
Octocat-spinner-32 slabs.c Make hash table algorithm selectable April 16, 2014
Octocat-spinner-32 slabs.h item locks now lock hash table buckets September 03, 2012
Octocat-spinner-32 solaris_priv.c Remove compilation errors from gcc on Solaris August 22, 2009
Octocat-spinner-32 stats.c Make hash table algorithm selectable April 16, 2014
Octocat-spinner-32 stats.h stats_prefix_* should not assume that all keys are zero-terminated January 03, 2009
Octocat-spinner-32 testapp.c fix for "variable length array folded to constant array as an extensi… December 09, 2013
Octocat-spinner-32 thread.c Make hash table algorithm selectable April 16, 2014
Octocat-spinner-32 timedrun.c bug237: Don't compute incorrect argc for timedrun January 11, 2012
Octocat-spinner-32 trace.h Backport binary TOUCH/GAT/GATQ commands September 27, 2011
Octocat-spinner-32 util.c Issue 221: Increment treats leading spaces as 0 September 03, 2011
Octocat-spinner-32 util.h Issue: #83: Refactor: use htonll or ntohll if the os provides them September 02, 2009
Octocat-spinner-32 version.pl Make autoversioning/spec gen work better. November 02, 2009
Octocat-spinner-32 version.sh Generate the memcached.spec along with the version number. August 29, 2009
README.md

Memcached

Dependencies

Environment

Linux

If using Linux, you need a kernel with epoll. Sure, libevent will work with normal select, but it sucks.

epoll isn't in Linux 2.4, but there's a backport at:

http://www.xmailserver.org/linux-patches/nio-improve.html

You want the epoll-lt patch (level-triggered).

Mac OS X

If you're using MacOS, you'll want libevent 1.1 or higher to deal with a kqueue bug.

Also, be warned that the -k (mlockall) option to memcached might be dangerous when using a large cache. Just make sure the memcached machines don't swap. memcached does non-blocking network I/O, but not disk. (it should never go to disk, or you've lost the whole point of it)

Website

Contributing

Want to contribute? Up-to-date pointers should be at:

Something went wrong with that request. Please try again.