Skip to content
a memcached proxy with energy and pep
C Python Perl Other
Find file
Latest commit 135282d Sep 7, 2010 @steveyen bug 2236 - don't printf unused dbpath anymore
the local sqlite/dbpath feature of libconflate was confusing,
so don't allude to it anymore in the -vvv output.

Change-Id: I849f72128b4859cf69f0b1a1a3cde6a93a99211c
Reviewed-on: http://review.northscale.com/2304
Reviewed-by: Steve Yen <steve.yen@gmail.com>
Tested-by: Steve Yen <steve.yen@gmail.com>
Failed to load latest commit information.
config using pandora config/m4 Jun 8, 2010
debian force use of bundled libconflate & libstrophe for binary packages Sep 29, 2009
devtools Implement 'noreply' option for update commands. (Tomash Brechko <toma… Feb 22, 2008
doc added downstream_conn_max config parsing Sep 14, 2010
m4 fixed cross-compilation with pandora_use_pipe Aug 4, 2010
scripts force use of bundled libconflate & libstrophe for binary packages Sep 29, 2009
t fixes tons of warnings produced on GNU/Linux Aug 6, 2010
win32 Win32 support Aug 4, 2010
.gitignore Add .libs to list of ignored files Sep 10, 2010
.gitmodules Changing submodule pointer to northscale github repository. Oct 4, 2009
.shipit Checking in changes prior to tagging of version 1.2.4. Changelog diff… Dec 6, 2007
AUTHORS autoconf/automake support from evan Jun 14, 2003
COPYING add a duplicate of the LICENSE file so GNU automake doesn't assume it… Aug 6, 2004
ChangeLog Changelog update from 1.2.8 fixes. Apr 11, 2009
Doxyfile Exclude testapp.c instead of internal_tests.c from docs. Apr 17, 2009
HACKING Remove reference to glib in the HACKING doc. Sep 29, 2009
LICENSE change from GPL to BSD. Jun 15, 2003
Makefile.am bug 2231 - MOXI_USE_LIBVBUCKET and MOXI_USE_LIBMEMCACHED Sep 7, 2010
NEWS auto* changes, update to version 1.0.2 Jun 14, 2003
README updated README Jul 8, 2010
TODO merge end of facebook branch into trunk, after copying the old trunk … Sep 3, 2006
TODO.moxi added stats todo items Jun 9, 2009
agent.h emitting histogram during "stats proxy timings" Sep 7, 2010
agent_config.c bug 2236 - don't printf unused dbpath anymore Sep 16, 2010
agent_ping.c cproxy_connect_downstream_conn() now calls mcs_connect() Sep 8, 2010
agent_stats.c stats for connect error cases Sep 14, 2010
assoc.c fixes tons of warnings produced on GNU/Linux Aug 6, 2010
assoc.h Refactor: moved the hash function from assoc.c to hash.c Mar 2, 2009
cJSON.c cJSON passes whitespace test Jul 28, 2010
cJSON.h cJSON passes whitespace test Jul 28, 2010
cache.c fixes tons of warnings produced on GNU/Linux Aug 6, 2010
cache.h Move to pandora build Aug 4, 2010
check_moxi.c allow tests to run memcached's main() with custom args and enabled fo… Sep 29, 2009
check_moxi_agent.c passing ./t/whitespace.t Jul 7, 2010
check_util.c run check tests with CK_ENV print mode Sep 29, 2009
check_work.c passing ./t/whitespace.t Jul 7, 2010
config_static.h Use the default automake rule to detect endian Sep 15, 2010
configure.ac Use the default automake rule to detect endian Sep 15, 2010
cproxy.c bug 2337, moved MOXI_BLOCKING_CONNECT definition Sep 14, 2010
cproxy.h stats for connect error cases Sep 14, 2010
cproxy_config.c mcmux compat needs cmd-line behavior Sep 15, 2010
cproxy_front.c fixes tons of warnings produced on GNU/Linux Aug 6, 2010
cproxy_multiget.c NULL_CONN sentinel when connect() fails Sep 8, 2010
cproxy_protocol_a.c mcmux-prefixed "A:HOST:PORT version" command Sep 15, 2010
cproxy_protocol_a2a.c NULL_CONN sentinel when connect() fails Sep 8, 2010
cproxy_protocol_a2b.c mcmux-prefixed "A:HOST:PORT version" command Sep 15, 2010
cproxy_protocol_b.c bug 2015 - more warnings around signed/unsigned Aug 12, 2010
cproxy_protocol_b2b.c bug 2345 - don't crash when some binary clients send in empty keys Sep 15, 2010
cproxy_stats.c stats for connect error cases Sep 14, 2010
daemon.c Move to pandora build Aug 4, 2010
genhash.c bug 2015 - more warnings around signed/unsigned Aug 12, 2010
genhash.h Replaced glib dependency with a custom hash table. Sep 2, 2009
genhash_int.h Replaced glib dependency with a custom hash table. Sep 2, 2009
globals.c start of the incr fix, rearranges a bunch, adds util, tests, etc Mar 19, 2009
hash.c fixes tons of warnings produced on GNU/Linux Aug 6, 2010
hash.h Refactor: moved the hash function from assoc.c to hash.c Mar 2, 2009
htgram.c htgram made consistent with memcached/moxi license Sep 7, 2010
htgram.h htgram made consistent with memcached/moxi license Sep 7, 2010
htgram_test.c histogram API Sep 7, 2010
items.c bug 2015 - more warnings around signed/unsigned Aug 12, 2010
items.h Don't expose the protocol used to the client api of the stats Apr 2, 2009
log.c too many newlines in logging Aug 11, 2010
log.h Migrate to using moxi log Jul 28, 2010
matcher.c Move to pandora build Aug 4, 2010
matcher.h matcher_check default_when_unstarted to avoid race Jun 11, 2009
mcs.c from zstored, 5 sec timeout during non-blocking connect() Sep 9, 2010
mcs.h windows needs EINPROGRESS, EISCONN Sep 13, 2010
memcached.c usage typo and width fix Sep 16, 2010
memcached.h mcmux-prefixed "A:HOST:PORT version" command Sep 15, 2010
memcached.spec.in Merged in 1.4.1 Sep 9, 2009
murmur_hash.c avoid libmemcached dependency Jun 8, 2010
protocol_binary.h updated moxi's PROTOCOL_BINARY enum's Jun 17, 2010
redirects.c passing ./t/whitespace.t Jul 7, 2010
redirects.h added redirects facility that allows us to mock various functions Sep 29, 2009
sizes.c fixes tons of warnings produced on GNU/Linux Aug 6, 2010
slabs.c all warnings ammeriolated Aug 10, 2010
slabs.h Don't expose the protocol used to the client api of the stats Apr 2, 2009
solaris_priv.c Move to pandora build Aug 4, 2010
stats.c passing ./t/whitespace.t Jul 7, 2010
stats.h stats_prefix_* should not assume that all keys are zero-terminated Jan 3, 2009
stdin_check.c fixes tons of warnings produced on GNU/Linux Aug 6, 2010
stdin_check.h Bug 1251: exiting when stdin closes Jun 9, 2010
testapp.c passing whitespace.pl tests again Aug 10, 2010
thread.c zstored_acquire/release_downstream_conn() pass make tests Sep 8, 2010
timedrun.c Merged in 1.4.1 Sep 10, 2009
trace.h Added trace.h to the list of sources to be included in the dist target Mar 2, 2009
util.c Move to pandora build Aug 4, 2010
util.h fixes tons of warnings produced on GNU/Linux Aug 6, 2010
version.sh Merge branch 'merge_of_1_4_1' Sep 29, 2009
work.c bug 2015 - more warnings around signed/unsigned Aug 12, 2010
work.h Move to pandora build Aug 4, 2010

README

moxi - a memcached/membase proxy with energy and pep

Dependencies:

   -- libevent, http://www.monkey.org/~provos/libevent/ (libevent-dev)

   -- libconflate, http://github.com/northscale/libconflate

   -- libvbucket, http://github.com/northscale/libvbucket

As a backwards-compatible (ketama/consistent-hashing) alternative to
libvbucket, you may instead use libmemcached instead of libvbucket...

   -- libmemcached, http://tangent.org/552/libmemcached.html

To use moxi against membase, however, you'll want libvbucket.

----------

To compile moxi (assuming you want libvbucket), after you got the
dependencies built and installed:

  ./config/autorun.sh
  ./configure
  make

For example, if libevent is installed in /opt/local, you'd do...

  ./config/autorun.sh
  ./configure --with-libevent=/opt/local
  make

----------

Using moxi:

To have moxi load a "vBucket" json configuration from a REST/HTTP
server, try...

  moxi http://host:port/url/of/vBucketServerMapJSON

Against a NorthScale server, for example, this would look like...

  moxi http://host:8080/pools/default/bucketsStreamingConfig/default

The above, also, is just shorthand (assuming you have no other flags
specified), for the following explicit command-line...

  moxi -z url=http://host:8080/url/of/vBucketServerMap.json

To get more command line usage info:

  moxi -h

----------

File-based configuration:

You may also provide a configuration file to moxi that holds a static
vBucket server map.  The file (such as vbucket1.cfg) would look
like...

  11211 = {
    "hashAlgorithm": "CRC",
    "numReplicas": 0,
    "serverList": ["memcached_svr1:11311"],
    "vBucketMap":
      [
        [0],
        [0]
      ]
  }

The above configuration would tell moxi to listen on port 11211, and
proxy to memcached_svr1:11311.  To use a static configuration file,
you would start moxi like...

  moxi -z ./vbucket1.cfg

The "./" path prefix is required so that moxi knows you're passing in
a config file.

----------

Tests:

  # To test that moxi still behaves like memcached
  # and passes all the tests that memcached passes...
  #
  make test

  # To test moxi in a simple proxy topology of...
  #  client <-> moxi <-> memcached
  #
  ./t/moxi.pl

  # To test moxi in a chained proxy topology of...
  #   client <-> moxi <-> moxi <-> moxi <-> memcached
  #
  ./t/moxi.pl chain

  # To test moxi in a fanout topology of...
  #   client <---> moxi <---> memcached
  #                       |-> memcached
  #                       |-> memcached
  #                       |-> memcached
  #
  ./t/moxi.pl fanout

  # To test moxi in a fanout and back in again topology of...
  #   client <---> moxi <---> moxi <-> memcached
  #            |-> moxi <-|
  #            |-> moxi <-|
  #            |-> moxi <-|
  #
  ./t/moxi.pl fanoutin

  # To test moxi proxy cases...
  #
  ./moxi -z 11333=localhost:11311 -t 1
  python t/moxi_mock.py

----------

For vbucket development, start the following...

  ruby ./t/rest_mock.rb

Then start a "pretend" memcached server...

  ./moxi -vvv -p 11311

Then...

  ./moxi -vvv -z url=http://localhost:4567/pools/default/bucketsStreaming/default -Z port_listen=11211

Then...

  telnet localhost 11211

----------

More notes:

If using Linux, you need a kernel with epoll (it's better than select()).

epoll isn't in Linux 2.4 yet, but there's a backport at:

    http://www.xmailserver.org/linux-patches/nio-improve.html

You want the epoll-lt patch (level-triggered).

If you're using MacOS, you'll want libevent 1.1 or higher to deal with
a kqueue bug.

Also, be warned that the -k (mlockall) option to memcached might be
dangerous when using a large cache.  Just make sure the memcached machines
don't swap.  memcached does non-blocking network I/O, but not disk.  (it
should never go to disk, or you've lost the whole point of it)

The memcached website is at:

    http://www.danga.com/memcached/

----------

Using moxi with libmemcached (instead of libvbucket)...

If you want to use moxi explicitly using ketama / consistent-hashing,
you'll need to compile moxi with libmemcached.

First, you would configure moxi differently at build-time...

  ./configure --enable-moxi-vbucket=no \
     CFLAGS=<libmemcached/include> \
     LDFLAGS=<libmemcahed/lib>

After building, moxi understands the following kind of command-line...

  moxi -z <port=<more-config>>

  moxi -z <port=<memcached_host:memcached_port(,*)>>

moxi will listen on the given port and accept connections from
upstream memcached clients.  It will forward requests to downstream
memcached servers running on memcached_host:memcached_port.  For
example...

  moxi -z 11211=my_memcached_server:11222

Above, moxi will listen on port 11211 and forward requests
to the memcached running on my_memcached_server that's listening
on port 11222.

You can list more than one memcached_host:memcached_port,
separated by commas.  For example...

  moxi -z 11211=memcached_1:11211,memcached_2:11211

The default downstream port is 11211, so you can also
just use...

  moxi -z 11211=memcached_1,memcached_2

If you have some memcached servers running on the same server,
but on different ports, you can put moxi in front of them
with something like...

  moxi -z 11211=server:11222,server:11233

Something went wrong with that request. Please try again.