Skip to content
This repository

a memcached proxy with energy and pep

bug 2236 - don't printf unused dbpath anymore

the local sqlite/dbpath feature of libconflate was confusing,
so don't allude to it anymore in the -vvv output.

Change-Id: I849f72128b4859cf69f0b1a1a3cde6a93a99211c
Reviewed-on: http://review.northscale.com/2304
Reviewed-by: Steve Yen <steve.yen@gmail.com>
Tested-by: Steve Yen <steve.yen@gmail.com>
latest commit 135282d725
Steve Yen authored September 07, 2010
Octocat-spinner-32 config using pandora config/m4 June 08, 2010
Octocat-spinner-32 debian force use of bundled libconflate & libstrophe for binary packages September 29, 2009
Octocat-spinner-32 devtools Implement 'noreply' option for update commands. (Tomash Brechko <toma… February 22, 2008
Octocat-spinner-32 doc added downstream_conn_max config parsing September 14, 2010
Octocat-spinner-32 m4 fixed cross-compilation with pandora_use_pipe August 04, 2010
Octocat-spinner-32 scripts force use of bundled libconflate & libstrophe for binary packages September 29, 2009
Octocat-spinner-32 t fixes tons of warnings produced on GNU/Linux August 06, 2010
Octocat-spinner-32 win32 Win32 support August 04, 2010
Octocat-spinner-32 .gitignore Add .libs to list of ignored files September 10, 2010
Octocat-spinner-32 .gitmodules Changing submodule pointer to northscale github repository. October 04, 2009
Octocat-spinner-32 .shipit
Octocat-spinner-32 AUTHORS autoconf/automake support from evan June 14, 2003
Octocat-spinner-32 COPYING add a duplicate of the LICENSE file so GNU automake doesn't assume it… August 06, 2004
Octocat-spinner-32 ChangeLog Changelog update from 1.2.8 fixes. April 11, 2009
Octocat-spinner-32 Doxyfile
Octocat-spinner-32 HACKING Remove reference to glib in the HACKING doc. September 29, 2009
Octocat-spinner-32 LICENSE change from GPL to BSD. June 15, 2003
Octocat-spinner-32 Makefile.am bug 2231 - MOXI_USE_LIBVBUCKET and MOXI_USE_LIBMEMCACHED September 07, 2010
Octocat-spinner-32 NEWS
Octocat-spinner-32 README updated README July 08, 2010
Octocat-spinner-32 TODO merge end of facebook branch into trunk, after copying the old trunk … September 03, 2006
Octocat-spinner-32 TODO.moxi added stats todo items June 09, 2009
Octocat-spinner-32 agent.h emitting histogram during "stats proxy timings" September 07, 2010
Octocat-spinner-32 agent_config.c bug 2236 - don't printf unused dbpath anymore September 16, 2010
Octocat-spinner-32 agent_ping.c cproxy_connect_downstream_conn() now calls mcs_connect() September 08, 2010
Octocat-spinner-32 agent_stats.c
Octocat-spinner-32 assoc.c fixes tons of warnings produced on GNU/Linux August 06, 2010
Octocat-spinner-32 assoc.h
Octocat-spinner-32 cJSON.c cJSON passes whitespace test July 28, 2010
Octocat-spinner-32 cJSON.h cJSON passes whitespace test July 28, 2010
Octocat-spinner-32 cache.c fixes tons of warnings produced on GNU/Linux August 06, 2010
Octocat-spinner-32 cache.h Move to pandora build August 04, 2010
Octocat-spinner-32 check_moxi.c allow tests to run memcached's main() with custom args and enabled fo… September 29, 2009
Octocat-spinner-32 check_moxi_agent.c passing ./t/whitespace.t July 07, 2010
Octocat-spinner-32 check_util.c run check tests with CK_ENV print mode September 29, 2009
Octocat-spinner-32 check_work.c passing ./t/whitespace.t July 07, 2010
Octocat-spinner-32 config_static.h Use the default automake rule to detect endian September 15, 2010
Octocat-spinner-32 configure.ac Use the default automake rule to detect endian September 15, 2010
Octocat-spinner-32 cproxy.c bug 2337, moved MOXI_BLOCKING_CONNECT definition September 14, 2010
Octocat-spinner-32 cproxy.h stats for connect error cases September 14, 2010
Octocat-spinner-32 cproxy_config.c mcmux compat needs cmd-line behavior September 15, 2010
Octocat-spinner-32 cproxy_front.c fixes tons of warnings produced on GNU/Linux August 06, 2010
Octocat-spinner-32 cproxy_multiget.c NULL_CONN sentinel when connect() fails September 08, 2010
Octocat-spinner-32 cproxy_protocol_a.c mcmux-prefixed "A:HOST:PORT version" command September 15, 2010
Octocat-spinner-32 cproxy_protocol_a2a.c
Octocat-spinner-32 cproxy_protocol_a2b.c mcmux-prefixed "A:HOST:PORT version" command September 15, 2010
Octocat-spinner-32 cproxy_protocol_b.c bug 2015 - more warnings around signed/unsigned August 12, 2010
Octocat-spinner-32 cproxy_protocol_b2b.c
Octocat-spinner-32 cproxy_stats.c stats for connect error cases September 14, 2010
Octocat-spinner-32 daemon.c Move to pandora build August 04, 2010
Octocat-spinner-32 genhash.c bug 2015 - more warnings around signed/unsigned August 12, 2010
Octocat-spinner-32 genhash.h Replaced glib dependency with a custom hash table. September 02, 2009
Octocat-spinner-32 genhash_int.h Replaced glib dependency with a custom hash table. September 02, 2009
Octocat-spinner-32 globals.c start of the incr fix, rearranges a bunch, adds util, tests, etc March 19, 2009
Octocat-spinner-32 hash.c fixes tons of warnings produced on GNU/Linux August 06, 2010
Octocat-spinner-32 hash.h Refactor: moved the hash function from assoc.c to hash.c March 02, 2009
Octocat-spinner-32 htgram.c htgram made consistent with memcached/moxi license September 07, 2010
Octocat-spinner-32 htgram.h htgram made consistent with memcached/moxi license September 07, 2010
Octocat-spinner-32 htgram_test.c histogram API September 07, 2010
Octocat-spinner-32 items.c bug 2015 - more warnings around signed/unsigned August 12, 2010
Octocat-spinner-32 items.h Don't expose the protocol used to the client api of the stats April 02, 2009
Octocat-spinner-32 log.c
Octocat-spinner-32 log.h Migrate to using moxi log July 28, 2010
Octocat-spinner-32 matcher.c Move to pandora build August 04, 2010
Octocat-spinner-32 matcher.h matcher_check default_when_unstarted to avoid race June 11, 2009
Octocat-spinner-32 mcs.c
Octocat-spinner-32 mcs.h windows needs EINPROGRESS, EISCONN September 13, 2010
Octocat-spinner-32 memcached.c usage typo and width fix September 16, 2010
Octocat-spinner-32 memcached.h mcmux-prefixed "A:HOST:PORT version" command September 15, 2010
Octocat-spinner-32 memcached.spec.in Merged in 1.4.1 September 09, 2009
Octocat-spinner-32 murmur_hash.c avoid libmemcached dependency June 08, 2010
Octocat-spinner-32 protocol_binary.h updated moxi's PROTOCOL_BINARY enum's June 17, 2010
Octocat-spinner-32 redirects.c passing ./t/whitespace.t July 07, 2010
Octocat-spinner-32 redirects.h added redirects facility that allows us to mock various functions September 29, 2009
Octocat-spinner-32 sizes.c fixes tons of warnings produced on GNU/Linux August 06, 2010
Octocat-spinner-32 slabs.c all warnings ammeriolated August 10, 2010
Octocat-spinner-32 slabs.h Don't expose the protocol used to the client api of the stats April 02, 2009
Octocat-spinner-32 solaris_priv.c Move to pandora build August 04, 2010
Octocat-spinner-32 stats.c passing ./t/whitespace.t July 07, 2010
Octocat-spinner-32 stats.h
Octocat-spinner-32 stdin_check.c fixes tons of warnings produced on GNU/Linux August 06, 2010
Octocat-spinner-32 stdin_check.h Bug 1251: exiting when stdin closes June 09, 2010
Octocat-spinner-32 testapp.c passing whitespace.pl tests again August 10, 2010
Octocat-spinner-32 thread.c zstored_acquire/release_downstream_conn() pass make tests September 08, 2010
Octocat-spinner-32 timedrun.c Merged in 1.4.1 September 09, 2009
Octocat-spinner-32 trace.h Added trace.h to the list of sources to be included in the dist target March 02, 2009
Octocat-spinner-32 util.c Move to pandora build August 04, 2010
Octocat-spinner-32 util.h fixes tons of warnings produced on GNU/Linux August 06, 2010
Octocat-spinner-32 version.sh Merge branch 'merge_of_1_4_1' September 29, 2009
Octocat-spinner-32 work.c bug 2015 - more warnings around signed/unsigned August 12, 2010
Octocat-spinner-32 work.h Move to pandora build August 04, 2010
README
moxi - a memcached/membase proxy with energy and pep

Dependencies:

   -- libevent, http://www.monkey.org/~provos/libevent/ (libevent-dev)

   -- libconflate, http://github.com/northscale/libconflate

   -- libvbucket, http://github.com/northscale/libvbucket

As a backwards-compatible (ketama/consistent-hashing) alternative to
libvbucket, you may instead use libmemcached instead of libvbucket...

   -- libmemcached, http://tangent.org/552/libmemcached.html

To use moxi against membase, however, you'll want libvbucket.

----------

To compile moxi (assuming you want libvbucket), after you got the
dependencies built and installed:

  ./config/autorun.sh
  ./configure
  make

For example, if libevent is installed in /opt/local, you'd do...

  ./config/autorun.sh
  ./configure --with-libevent=/opt/local
  make

----------

Using moxi:

To have moxi load a "vBucket" json configuration from a REST/HTTP
server, try...

  moxi http://host:port/url/of/vBucketServerMapJSON

Against a NorthScale server, for example, this would look like...

  moxi http://host:8080/pools/default/bucketsStreamingConfig/default

The above, also, is just shorthand (assuming you have no other flags
specified), for the following explicit command-line...

  moxi -z url=http://host:8080/url/of/vBucketServerMap.json

To get more command line usage info:

  moxi -h

----------

File-based configuration:

You may also provide a configuration file to moxi that holds a static
vBucket server map.  The file (such as vbucket1.cfg) would look
like...

  11211 = {
    "hashAlgorithm": "CRC",
    "numReplicas": 0,
    "serverList": ["memcached_svr1:11311"],
    "vBucketMap":
      [
        [0],
        [0]
      ]
  }

The above configuration would tell moxi to listen on port 11211, and
proxy to memcached_svr1:11311.  To use a static configuration file,
you would start moxi like...

  moxi -z ./vbucket1.cfg

The "./" path prefix is required so that moxi knows you're passing in
a config file.

----------

Tests:

  # To test that moxi still behaves like memcached
  # and passes all the tests that memcached passes...
  #
  make test

  # To test moxi in a simple proxy topology of...
  #  client <-> moxi <-> memcached
  #
  ./t/moxi.pl

  # To test moxi in a chained proxy topology of...
  #   client <-> moxi <-> moxi <-> moxi <-> memcached
  #
  ./t/moxi.pl chain

  # To test moxi in a fanout topology of...
  #   client <---> moxi <---> memcached
  #                       |-> memcached
  #                       |-> memcached
  #                       |-> memcached
  #
  ./t/moxi.pl fanout

  # To test moxi in a fanout and back in again topology of...
  #   client <---> moxi <---> moxi <-> memcached
  #            |-> moxi <-|
  #            |-> moxi <-|
  #            |-> moxi <-|
  #
  ./t/moxi.pl fanoutin

  # To test moxi proxy cases...
  #
  ./moxi -z 11333=localhost:11311 -t 1
  python t/moxi_mock.py

----------

For vbucket development, start the following...

  ruby ./t/rest_mock.rb

Then start a "pretend" memcached server...

  ./moxi -vvv -p 11311

Then...

  ./moxi -vvv -z url=http://localhost:4567/pools/default/bucketsStreaming/default -Z port_listen=11211

Then...

  telnet localhost 11211

----------

More notes:

If using Linux, you need a kernel with epoll (it's better than select()).

epoll isn't in Linux 2.4 yet, but there's a backport at:

    http://www.xmailserver.org/linux-patches/nio-improve.html

You want the epoll-lt patch (level-triggered).

If you're using MacOS, you'll want libevent 1.1 or higher to deal with
a kqueue bug.

Also, be warned that the -k (mlockall) option to memcached might be
dangerous when using a large cache.  Just make sure the memcached machines
don't swap.  memcached does non-blocking network I/O, but not disk.  (it
should never go to disk, or you've lost the whole point of it)

The memcached website is at:

    http://www.danga.com/memcached/

----------

Using moxi with libmemcached (instead of libvbucket)...

If you want to use moxi explicitly using ketama / consistent-hashing,
you'll need to compile moxi with libmemcached.

First, you would configure moxi differently at build-time...

  ./configure --enable-moxi-vbucket=no \
     CFLAGS=<libmemcached/include> \
     LDFLAGS=<libmemcahed/lib>

After building, moxi understands the following kind of command-line...

  moxi -z <port=<more-config>>

  moxi -z <port=<memcached_host:memcached_port(,*)>>

moxi will listen on the given port and accept connections from
upstream memcached clients.  It will forward requests to downstream
memcached servers running on memcached_host:memcached_port.  For
example...

  moxi -z 11211=my_memcached_server:11222

Above, moxi will listen on port 11211 and forward requests
to the memcached running on my_memcached_server that's listening
on port 11222.

You can list more than one memcached_host:memcached_port,
separated by commas.  For example...

  moxi -z 11211=memcached_1:11211,memcached_2:11211

The default downstream port is 11211, so you can also
just use...

  moxi -z 11211=memcached_1,memcached_2

If you have some memcached servers running on the same server,
but on different ports, you can put moxi in front of them
with something like...

  moxi -z 11211=server:11222,server:11233

Something went wrong with that request. Please try again.