jmhodges / vpork forked from trav/vpork

Distributed hash table performance tool

This URL has Read+Write access

vpork /
name age
history
message
file .gitignore Thu May 07 09:24:57 -0700 2009 Use blocking inserts, gives better read perform... [johanoskarsson]
file LICENSE Tue Mar 31 11:34:41 -0700 2009 Correct the config file name, more cleanups / l... [trav]
file README Mon Jun 01 19:35:12 -0700 2009 Small readme tweaks [trav]
directory configs/ Mon Jun 01 23:27:40 -0700 2009 Add memory client. [trav]
directory lib/ Mon Jun 01 20:33:16 -0700 2009 Update voldemort version to latest. Adapt conf... [trav]
file run-groovy.sh Tue Jun 02 16:36:03 -0700 2009 Allocate mo-memory to our client [trav]
directory src/ Thu Jun 18 15:20:35 -0700 2009 voldemort logging should be at info, not debug [trav]
directory templates/ Tue May 05 07:39:35 -0700 2009 Refactored to allow make it easier to add Cassa... [johanoskarsson]
file vpork.sh Mon Mar 30 13:27:33 -0700 2009 Init [trav]
README
VPork

Introduction
VPork is a utility for load-testing various distributed databases.


Capabilities
VPork was written with a fairly specific use-case in mind, but provides a
fairly general pattern of access:

    * A single JVM is started, with any number of client threads
    * Each thread executes for a given number of iterations
    * For each iteration, the thread can read an existing record, and/or create
      a new one
    * The probability of each read/write is configurable
    * The location of where reads happen is configurable (by default, it
      reads the most recently written records, trailing off to less frequent
      reads of writes which occurred long ago)


Pre-Requisites
These steps already assume that you have distributed database up and running in
a cluster.

VPork also requires groovy, which can be obtained at groovy.codehaus.org



** Install / Configure **
Download vpork:

$ git clone git://github.com/trav/vpork.git


For Voldemort:
Install an appropriate stores.xml file. VPork ships with a
file in templates/perfstores.xml, which contains a store called 'bytez'
which is used by VPork. You can copy perfstores.xml directly to your
configuration directory or make similar changes to your existing configs:

$ cp templates/perfstores.xml /mnt/project-voldemort/config/single_node_cluster/config/stores.xml

$ ./vpork.sh configs/voldemort/30-thread-pork.groovy configs/voldemort/nodes.conf


For Cassandra:
You can use the default storage settings for Cassandra.
Just edit the Vpork nodes.conf file to include the nodes in your cluster.

$ ./vpork.sh configs/cassandra/30-thread-pork.groovy configs/cassandra/nodes.conf


If everything is working, you should see results in results/30-thread-pork-0



** Log files **
The output of VPork is put into a test-versioned directory with snapshots of your Voldemort configurations:

$ cat results/30-thread-pork-0/progress.log




Mar 31, 2009 6:21:17 PM - Writes:
Mar 31, 2009 6:21:17 PM -   Num Writes:           159855
Mar 31, 2009 6:21:17 PM -   Write Failures:       0
Mar 31, 2009 6:21:17 PM -   Write Latency:        85.94 ms
Mar 31, 2009 6:21:19 PM -   Write Latency (%99):  319.00 ms
Mar 31, 2009 6:21:19 PM -   Bytes Written:        3658.79 MB
Mar 31, 2009 6:21:19 PM -   Thread w/Throughput:  0.27 KB / ms
Mar 31, 2009 6:21:19 PM -   Total w/Throughput:   24.29 KB / ms
Mar 31, 2009 6:21:19 PM - 
Mar 31, 2009 6:21:19 PM - Reads:
Mar 31, 2009 6:21:19 PM -   Num Read:             19872
Mar 31, 2009 6:21:19 PM -   Read Failures:        0
Mar 31, 2009 6:21:19 PM -   Read Latency:         70.36 ms
Mar 31, 2009 6:21:19 PM -   Read Latency (%99):   298.00 ms
Mar 31, 2009 6:21:19 PM -   Read Not Found:       12 (%0.06)
Mar 31, 2009 6:21:19 PM -   Bytes Read:           453.94 MB
Mar 31, 2009 6:21:19 PM -   Thread r/Throughput:  0.33 KB / ms
Mar 31, 2009 6:21:19 PM -   Total r/Throughput:   3.01 KB / ms