public
Description: a memcached proxy with energy and pep
Homepage:
Clone URL: git://github.com/steveyen/moxi.git
moxi /
name age message
file .gitignore Tue Sep 29 01:43:28 -0700 2009 Merge branch 'merge_of_1_4_1' [alk]
file .gitmodules Fri Oct 02 09:38:33 -0700 2009 Updated submodule configuration to use northsca... [Matt Ingenthron]
file .shipit Thu Dec 06 01:38:49 -0800 2007 Checking in changes prior to tagging of version... [dormando]
file AUTHORS Fri Jun 13 19:27:53 -0700 2003 autoconf/automake support from evan git-svn-i... [Brad Fitzpatrick]
file BUILD Thu Jul 31 16:37:04 -0700 2003 better build instructions git-svn-id: http://... [Brad Fitzpatrick]
file COPYING Thu Aug 05 20:04:15 -0700 2004 add a duplicate of the LICENSE file so GNU auto... [Brad Fitzpatrick]
file ChangeLog Sat Apr 11 21:49:32 -0700 2009 Changelog update from 1.2.8 fixes. [dormando]
file Doxyfile Fri Apr 17 16:27:17 -0700 2009 Exclude testapp.c instead of internal_tests.c f... [dustin]
file HACKING Wed Sep 02 13:53:34 -0700 2009 Remove reference to glib in the HACKING doc. [dustin]
file LICENSE Thu Aug 05 20:04:15 -0700 2004 add a duplicate of the LICENSE file so GNU auto... [Brad Fitzpatrick]
file Makefile.am Fri Oct 02 07:43:18 -0700 2009 make sure that redirects.h is included into dis... [alk]
file NEWS Sat Jun 14 16:07:09 -0700 2003 auto* changes, update to version 1.0.2 git-sv... [Brad Fitzpatrick]
file README Fri Apr 24 22:05:55 -0700 2009 README grammar fixes [steveyen]
file TODO Sat Sep 02 20:18:26 -0700 2006 merge end of facebook branch into trunk, after ... [Brad Fitzpatrick]
file TODO.moxi Tue Jun 09 09:19:48 -0700 2009 added stats todo items [steveyen]
file agent.h Mon Sep 28 06:53:16 -0700 2009 added redirects facility that allows us to mock... [alk]
file agent_config.c Tue Sep 29 01:49:39 -0700 2009 Merge remote branch 'northscale/master' [alk]
file agent_ping.c Thu Aug 13 12:51:07 -0700 2009 move to TCP_NODELAY option with libmemcached, f... [alk]
file agent_stats.c Mon Sep 28 06:53:16 -0700 2009 added redirects facility that allows us to mock... [alk]
file assoc.c Tue Mar 24 12:59:47 -0700 2009 "stats reset" should reset eviction counters as... [Trond Norbye]
file assoc.h Mon Mar 02 01:15:34 -0800 2009 Refactor: moved the hash function from assoc.c ... [Trond Norbye]
file autogen.sh Fri Oct 09 13:46:06 -0700 2009 When doing a recursive clean, also ensure submo... [Matt Ingenthron]
file cache.c Fri Apr 17 16:15:52 -0700 2009 Create a generic cache for objects of same size... [Trond Norbye]
file cache.h Fri Apr 17 16:15:58 -0700 2009 Added documentation on the cache interface [Trond Norbye]
file check_moxi.c Mon Sep 28 06:52:46 -0700 2009 allow tests to run memcached's main() with cust... [alk]
file check_moxi_agent.c Fri Oct 02 06:25:17 -0700 2009 workaround some weird behaviour (bug?) of osx's... [alk]
file check_util.c Mon Sep 28 06:51:35 -0700 2009 run check tests with CK_ENV print mode [alk]
file check_work.c Mon Sep 28 06:52:46 -0700 2009 allow tests to run memcached's main() with cust... [alk]
file configure.ac Thu Oct 15 19:38:46 -0700 2009 Libevent detection should be built just like ev... [Matt Ingenthron]
file cproxy.c Tue Sep 29 01:49:39 -0700 2009 Merge remote branch 'northscale/master' [alk]
file cproxy.h Fri Sep 18 02:17:37 -0700 2009 implemented mcache iteration function [alk]
file cproxy_config.c Wed Sep 02 13:07:49 -0700 2009 Replaced glib dependency with a custom hash tab... [dustin]
file cproxy_front.c Fri Sep 18 02:17:37 -0700 2009 implemented mcache iteration function [alk]
file cproxy_multiget.c Fri Sep 18 02:17:37 -0700 2009 use genhash_update instead of genhash_store. Th... [alk]
file cproxy_protocol.c Tue Sep 29 01:49:39 -0700 2009 Merge remote branch 'northscale/master' [alk]
file cproxy_protocol_a2a.c Wed Sep 02 13:07:49 -0700 2009 Replaced glib dependency with a custom hash tab... [dustin]
file cproxy_protocol_a2b.c Tue Sep 29 01:49:39 -0700 2009 Merge remote branch 'northscale/master' [alk]
file cproxy_stats.c Fri Sep 18 02:17:37 -0700 2009 added 'added_at' field to struct key_stats, so ... [alk]
file daemon.c Tue Jan 20 11:23:14 -0800 2009 Build fixes for ubuntu 8.10/64. gcc seems to h... [dustin]
directory debian/ Wed Sep 23 14:01:38 -0700 2009 force use of bundled libconflate & libstrophe f... [alk]
directory devtools/ Thu Feb 21 20:57:41 -0800 2008 Implement 'noreply' option for update commands.... [Tomash Brechko]
directory doc/ Tue Sep 29 01:43:28 -0700 2009 Merge branch 'merge_of_1_4_1' [alk]
file genhash.c Wed Sep 02 13:07:49 -0700 2009 Replaced glib dependency with a custom hash tab... [dustin]
file genhash.h Wed Sep 02 13:07:49 -0700 2009 Replaced glib dependency with a custom hash tab... [dustin]
file genhash_int.h Wed Sep 02 13:07:49 -0700 2009 Replaced glib dependency with a custom hash tab... [dustin]
file globals.c Thu Mar 19 01:52:53 -0700 2009 start of the incr fix, rearranges a bunch, adds... [Brad Fitzpatrick]
file hash.c Mon Mar 02 01:15:34 -0800 2009 Refactor: moved the hash function from assoc.c ... [Trond Norbye]
file hash.h Mon Mar 02 01:15:34 -0800 2009 Refactor: moved the hash function from assoc.c ... [Trond Norbye]
file items.c Wed Sep 09 21:31:42 -0700 2009 Merged in 1.4.1 [dustin]
file items.h Thu Apr 02 12:24:08 -0700 2009 Don't expose the protocol used to the client ap... [Trond Norbye]
submodule libconflate - 5717969 Tue Oct 13 13:48:11 -0700 2009 Submodule update, libconflate. [Matt Ingenthron]
directory libmemcached-0.30/ Tue Oct 13 19:15:48 -0700 2009 Enums should be used consistently (needed for I... [Matt Ingenthron]
file matcher.c Tue Jun 23 20:32:34 -0700 2009 matcher needs strdup(spec) and check/test matcher [steveyen]
file matcher.h Thu Jun 11 19:51:04 -0700 2009 matcher_check default_when_unstarted to avoid race [steveyen]
file memcached.c Tue Sep 29 01:43:28 -0700 2009 Merge branch 'merge_of_1_4_1' [alk]
file memcached.h Tue Sep 29 01:43:28 -0700 2009 Merge branch 'merge_of_1_4_1' [alk]
file memcached.spec.in Wed Sep 09 21:31:42 -0700 2009 Merged in 1.4.1 [dustin]
file memcached_dtrace.d Wed Sep 09 21:31:42 -0700 2009 Merged in 1.4.1 [dustin]
directory patches/ Sat Jul 04 14:01:35 -0700 2009 cleanup of several things on SUBDIR build of li... [Matt Ingenthron]
file protocol_binary.h Mon Jul 13 23:48:03 -0700 2009 New binary protocol error for bad stored value ... [dustin]
file redirects.c Mon Sep 28 06:53:16 -0700 2009 added redirects facility that allows us to mock... [alk]
file redirects.h Mon Sep 28 06:53:16 -0700 2009 added redirects facility that allows us to mock... [alk]
directory scripts/ Wed Sep 23 14:01:38 -0700 2009 force use of bundled libconflate & libstrophe f... [alk]
file sizes.c Tue Mar 10 12:59:07 -0700 2009 Created a tool to show us the sizes of various ... [dustin]
file slabs.c Wed Sep 09 21:31:42 -0700 2009 Merged in 1.4.1 [dustin]
file slabs.h Thu Apr 02 12:24:08 -0700 2009 Don't expose the protocol used to the client ap... [Trond Norbye]
file solaris_priv.c Wed Sep 09 21:31:42 -0700 2009 Merged in 1.4.1 [dustin]
file stats.c Wed Sep 09 21:31:42 -0700 2009 Merged in 1.4.1 [dustin]
file stats.h Sat Jan 03 00:09:11 -0800 2009 stats_prefix_* should not assume that all keys ... [Trond Norbye]
directory t/ Tue Sep 29 01:43:28 -0700 2009 Merge branch 'merge_of_1_4_1' [alk]
file testapp.c Wed Sep 09 21:31:42 -0700 2009 Merged in 1.4.1 [dustin]
file thread.c Tue Sep 29 01:39:25 -0700 2009 pass binding protocol to dispatch_conn_new thi... [alk]
file timedrun.c Wed Sep 09 21:31:42 -0700 2009 Merged in 1.4.1 [dustin]
file trace.h Mon Mar 02 11:59:11 -0800 2009 Added trace.h to the list of sources to be incl... [Trond Norbye]
file util.c Wed Sep 09 21:31:42 -0700 2009 Merged in 1.4.1 [dustin]
file util.h Wed Sep 09 21:31:42 -0700 2009 Merged in 1.4.1 [dustin]
file version.sh Tue Sep 29 01:43:28 -0700 2009 Merge branch 'merge_of_1_4_1' [alk]
file work.c Sat Jun 20 00:37:01 -0700 2009 check_work and check_moxi_agetn [steveyen]
file work.h Sat Jun 20 00:37:01 -0700 2009 check_work and check_moxi_agetn [steveyen]
README
moxi - a memcached proxy with energy and pep

Dependencies:

   -- libevent, http://www.monkey.org/~provos/libevent/ (libevent-dev)
   -- libmemcached, http://tangent.org/552/libmemcached.html

To compile moxi, after you got the dependencies:

  ./autogen.sh
  ./configure
  make

For example, if libevent is installed in /opt/local, you'd do...

  ./autogen.sh
  ./configure --with-libevent=/opt/local
  make

Using moxi:

In moxi terminology, "upstream" are the memcached clients,
and "downstream" are the memcached servers.

  moxi -z <port=<memcached_host:memcached_port(,*)>>

moxi will listen on the given port and accept connections from
upstream memcached clients.  It will forward requests to downstream
memcached servers running on memcached_host:memcached_port.  For
example...

  moxi -z 11211=my_memcached_server:11222

Above, moxi will listen on port 11211 and forward requests
to the memcached running on my_memcached_server that's listening
on port 11222.

You can list more than one memcached_host:memcached_port,
separated by commas.  For example...

  moxi -z 11211=memcached_1:11211,memcached_2:11211

The default downstream port is 11211, so you can also
just use...

  moxi -z 11211=memcached_1,memcached_2

If you have some memcached servers running on the same server,
but on different ports, you can put moxi in front of them
with something like...

  moxi -z 11211=server:11222,server:11233

To get more command line usage info:

  moxi -h

Tests:

  # To test that moxi still behaves like memcached
  # and passes all the tests that memcached passes...
  #
  make test

  # To test moxi in a simple proxy topology of...
  #  client <-> moxi <-> memcached
  #
  ./t/moxi.pl

  # To test moxi in a chained proxy topology of...
  #   client <-> moxi <-> moxi <-> moxi <-> memcached
  #
  ./t/moxi.pl chain

  # To test moxi in a fanout topology of...
  #   client <---> moxi <---> memcached
  #                       |-> memcached
  #                       |-> memcached
  #                       |-> memcached
  #
  ./t/moxi.pl fanout

  # To test moxi in a fanout and back in again topology of...
  #   client <---> moxi <---> moxi <-> memcached
  #            |-> moxi <-|
  #            |-> moxi <-|
  #            |-> moxi <-|
  #
  ./t/moxi.pl fanoutin

  # To test moxi proxy cases...
  #
  ./moxi -z 11333=localhost:11311 -t 1
  python t/moxi_mock.py

More notes:

If using Linux, you need a kernel with epoll.  Sure, libevent will
work with normal select, but it sucks.

epoll isn't in Linux 2.4 yet, but there's a backport at:

    http://www.xmailserver.org/linux-patches/nio-improve.html

You want the epoll-lt patch (level-triggered).

If you're using MacOS, you'll want libevent 1.1 or higher to deal with
a kqueue bug.

Also, be warned that the -k (mlockall) option to memcached might be
dangerous when using a large cache.  Just make sure the memcached machines
don't swap.  memcached does non-blocking network I/O, but not disk.  (it
should never go to disk, or you've lost the whole point of it)

The memcached website is at:

    http://www.danga.com/memcached/