ENSIGN CRUSHER
Switch branches/tags
Nothing to show
Clone or download
dormando fix sock leak and improve pacing
missed removing old line of socket code which was doubling number of
sockets initialized (though only half were used).

use the real clock for pacing rather than relying solely on evtimer.

evtimer was very, very wobbly, especially at low times. now we attempt
to sleep for the right amount of time, and adjust up/down based on real
timings.

it's still a little wrong: every entry into the timer function means
we've already sent a blast of data, so the only option is to decide how
long to attempt to sleep before doing it again. this averages out close
to correct over a few runs, so keeping pacing down to tens of
milliseconds is probably necessary.
Latest commit 23d6d87 Oct 3, 2018

README

An extremely rough but extremely damaging benchmark utility for memcached.

Does not use a client library, does not use threads (though it will). Uses
event based sockets via libevent.

Building
--------

$ (install libevent + headers)
$ ./compile

Usage
-----

Look in the conf/ directory for example configuration files.

$ ./mc-crusher ./conf/loadconf [host] [port]
defaults:
host: 127.0.0.1
port: 11211

Running
-------

This tool is extremely rough.

First, run it for a few seconds with a config like "conf/loadconf", this will
seed the instance with some values to fetch later. Now, kill it.

You can also use the bench-warmer.pl tool to preload some keys.

Pick a config file, and start it as above. You should start a fresh memcached,
and run the "sample" script that comes with mc-crusher. mc-crusher makes no
attempt to care about what it does. It flails tiredlessly against the fanged
defenses of the cache daemon.

Modify bench-sample to print what you are most interested in examining.

You can change the "default" destination ip/port by adding extra arguments:
$ ./mc-crusher ./conf/loadconf 192.168.1.1 11211

Configuration
-------------

mc-crusher reads a configuration file then executes. These configurations
describe a "type" of connection, one per line.

Each "type" can spawn N connections. This allows you to mix setters and
getters, getters of different sizes, binprot + asciiprot. It's very limited
right now, but I wanted to build this in as a base.

Config Options
--------------

send : defines what function to use to send requests to memcached
 - ascii_get : one get per request via asciiprot
 - ascii_set : one set per request via asciiprot
 - ascii_delete : one delete per request via asciiprot
 - ascii_mget : multiget test via asciiprot
 - bin_get : one get per req via binprot
 - bin_getq : endless streaming multiget from hell
 - bin_set : one set per req via binprot
 - bin_setq : unleash cthulu upon the cache_lock

recv : same, but for received data
 - blind_recv : mindlessly slurp any responses without inspecting them

mget_count : set this to the number of keys to fetch per get in ascii_mget

key_prefix : a string to prefix before each key's number (default 'foo')

value_size : size of the value to set

expire : the expiration value (default 0)

flags : client flags (raw number, default 0)

value : define a value by hand. Must be shortish and a string.

host : define a host to connect to

port : define a port to connect to

key_prealloc : pre-create key strings in memory to reduce bench cpu usage

key_count : number of keys to iterate across. use with key_prealloc=0 to
iterate over large numbers of keys

key_randomize : if using prealloc'ed keys, shuffle the list after generating

usleep : fire one write event (up to pipeline count) with a microsecond sleep
after. the sleep is per-connection (so conns=2 will write twice per usleep)

stop_after : stop all conns for this test after N writes have been made.

Caveats
-------

- I bundled the protocol_binary.h header. It will need to be copied from the
  primary repo for updates.

- Does not make any attempt to safely parse the config file. If you don't type
  it exactly right you will end up with bizarre failures.

- It only works well with small values

- It makes no attempt to reconcile with errors in the protocol, and can break
  if memcached throws errors. It will also stop if the connections are closed.

- It's a ton of fun!

Future Features
---------------

A short list of things I intend to change or add:

- Fix the command generators to have a *little* error handling

- Switch to writev and use iovectors + pregenerated keys for almost everything

- Add back some routines which are sprintf only, for doing tests with
  extremely wide keyspaces where performance is less of an issue.

- Add a "run_every" option which fires a conn every N microseconds instead of
  endlessly.

- Add a "timer" option which times the commands it runs and periodically dumps
  a histogram of response times. Mainly to be used with "run_every"

- Add missing protocol commands. *_incr, *_decr, *_delete, etc.

- Make the iteration loop per-connection, and randomizeable.

- Add commands to iterate over different value sizes

- Add multithreading to ease management.

- Allow running timer conns in their own thread (or just define if a conn
  template should use a thread, then define conns=1?)

- Bundle a better perl util for printing stats

- Write/bundle a better util for running iterative benchmarks.