initial commit

I kid, this was actually 20 commits but I'm squashing it for the first push
onto github.

LICENSE

@@ -0,0 +1,30 @@
+Copyright (c) 2011 Dormando.
+All rights reserved.
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are
+met:
+
+    * Redistributions of source code must retain the above copyright
+notice, this list of conditions and the following disclaimer.
+
+    * Redistributions in binary form must reproduce the above
+copyright notice, this list of conditions and the following disclaimer
+in the documentation and/or other materials provided with the
+distribution.
+
+    * Neither the name of the Danga Interactive nor the names of its
+contributors may be used to endorse or promote products derived from
+this software without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

README

@@ -0,0 +1,124 @@
+An extremely rough but extremely damaging benchmark utility for memcached.
+
+Does not use a client library, does not use threads (though it will). Uses
+event based sockets via libevent.
+
+Building
+--------
+
+$ (install libevent + headers)
+$ ./compile
+
+Usage
+-----
+
+Look in the conf/ directory for example configuration files.
+
+$ ./mc-crusher ./conf/loadconf
+
+Running
+-------
+
+This guy is extremely rough. It has hardcoded values such as a 200,000 item
+iteration loop. I'm releasing it early for shiggles.
+
+First, run it for a few seconds with a config like "conf/loadconf", this will
+seed the instance with some values to fetch later. Now, kill it.
+
+Pick a config file, and start it as above. You should start a fresh memcached,
+and run the "sample" script that comes with mc-crusher. mc-crusher makes no
+attempt to care about what it does. It flails tiredlessly against the fanged
+defenses of the cache daemon.
+
+Modify bench-sample to print what you are most interested in examining. Start
+multiple instances of mc-crusher if you think memcached can take more. In the
+future threads will be used for user convenience.
+
+Configuration
+-------------
+
+mc-crusher reads a configuration file then executes. These configurations
+describe a "type" of connection, one per line.
+
+Each "type" can spawn N connections. This allows you to mix setters and
+getters, getters of different sizes, binprot + asciiprot. It's very limited
+right now, but I wanted to build this in as a base.
+
+Config Options
+--------------
+
+send : defines what function to use to send requests to memcached
+ - ascii_get : one get per request via asciiprot
+ - ascii_set : one set per request via asciiprot
+ - ascii_mget : multiget test via asciiprot
+ - bin_get : one get per req via binprot
+ - bin_getq : endless streaming multiget from hell
+ - bin_set : one set per req via binprot
+ - bin_setq : unleash cthulu upon the cache_lock
+
+recv : same, but for received data
+ - blind_recv : mindlessly slurp any responses without inspecting them
+
+mget_count : set this to the number of keys to fetch per get in ascii_mget
+
+key_prefix : a string to prefix before each key's number (default 'foo')
+
+value_size : size of the value to set
+
+value : define a value by hand. Must be shortish and a string.
+
+Caveats
+-------
+
+- I bundled the protocol_binary.h header because I'm an asshole, deal with it.
+
+- Hardcoded to hit a localhost instance. Just recompile if you want it
+elsehwere.
+
+- Does not make any attempt to safely parse the config file. If you don't type
+  it exactly right you will end up with bizarre failures.
+
+- It is missing many features I intend to add and use; however as is it was
+  able to verify some lock scaling patches I worked on.
+
+- It mindlessly iterates through 200,000 keys.
+
+- It only works well with small values
+
+- It makes no attempt to reconcile with errors in the protocol, and can break
+  if memcached throws errors. It will also stop if the connections are closed.
+
+- It's a ton of fun!
+
+Future Features
+---------------
+
+A short list of things I intend to change or add:
+
+- Fix the command generators to have a *little* error handling
+
+- Switch to writev and use iovectors + pregenerated keys for almost everything
+
+- Add back some routines which are sprintf only, for doing tests with
+  extremely wide keyspaces where performance is less of an issue.
+
+- Add a "run_every" option which fires a conn every N microseconds instead of
+  endlessly.
+
+- Add a "timer" option which times the commands it runs and periodically dumps
+  a histogram of response times. Mainly to be used with "run_every"
+
+- Add missing protocol commands. *_incr, *_decr, *_delete, etc.
+
+- Make the iteration loop per-connection, and randomizeable.
+
+- Add commands to iterate over different value sizes
+
+- Add multithreading to ease management.
+
+- Allow running timer conns in their own thread (or just define if a conn
+  template should use a thread, then define conns=1?)
+
+- Bundle a better perl util for printing stats
+
+- Write/bundle a better util for running iterative benchmarks.

bench-sample

@@ -0,0 +1,61 @@
+#!/usr/bin/perl
+# public domain
+# benchmark sampler by dormando.
+# It says "redis" in a few places because I wrote this for that redis v.
+# memcached benchmark and I've decided to write thi scomment instead of edit
+# the code right now.
+
+use warnings;
+use strict;
+use IO::Socket::INET;
+use Time::HiRes;
+
+my $type        = $ARGV[0];
+my $SAMPLE_TIME = $ARGV[1] || 2;
+
+my $addr = $type eq 'redis' ? "127.0.0.1:6379" : "127.0.0.1:11211";
+my $s = connect_to($addr);
+die $@ unless $s;
+
+my $one      = $type eq 'redis' ? redis_info($s) : mc_info($s);
+my $one_time = Time::HiRes::time;
+
+sleep $SAMPLE_TIME;
+
+my $two      = $type eq 'redis' ? redis_info($s) : mc_info($s);
+my $two_time = Time::HiRes::time;
+
+my $elapsed = $two_time - $one_time;
+my $per_sec = ($two - $one) / $elapsed;
+print "$type per-second average ($SAMPLE_TIME): $per_sec\n";
+
+sub redis_info {
+    my $s = shift;
+    print $s "info\r\n";
+    while (my $line = <$s>) {
+        if ($line =~ m/total_commands_processed:(\d+)/) {
+            return $1;
+        }
+    }
+}
+
+sub mc_info {
+    my $s = shift;
+    print $s "stats\r\n";
+    my $get = -1;
+    my $set = -1;
+    while (my $line = <$s>) {
+        last if $line =~ m/^END/;
+        if ($line =~ m/^STAT cmd_get (\d+)/) {
+            $get = $1;
+        } elsif ($line =~ m/STAT cmd_set (\d+)/) {
+            $set = $1;
+        }
+    }
+    # Just comparing the combined stats...
+    return $get + $set;
+}
+
+sub connect_to {
+    return IO::Socket::INET->new(PeerAddr => $_[0], Timeout => 3);
+}

compile

@@ -0,0 +1,4 @@
+#!/bin/bash
+# I violate the laws of reason.
+rm -f mc-crusher
+gcc -g -O2 -o mc-crusher -levent ./mc-crusher.c

conf/asciiconf

@@ -0,0 +1 @@
+send=ascii_get,recv=blind_read,conns=50,key_prefix=foobar

conf/binconf

@@ -0,0 +1 @@
+send=bin_setq,recv=blind_read,conns=100,key_prefix=foobar

conf/loadconf

@@ -0,0 +1 @@
+send=ascii_set,recv=blind_read,conns=5,key_prefix=foobar,value_size=2,value=hi

conf/toastconf

@@ -0,0 +1,2 @@
+send=ascii_mget,recv=blind_read,conns=50,mget_count=5,key_prefix=foobar
+send=ascii_set,recv=blind_read,conns=1,key_prefix=foobar,value_size=2