Skip to content
WORK IN PROGRESS Mad Science is a testing / benchmarking framework in python targeting server configuration benchmarking. Mad Science has this basic workflow: Restart services so you start from a known status, make one configuration change, restart services to apply the change, benchmark the server, record the results. Rinse, repeat
Python
Find file
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Failed to load latest commit information.
Beaker.py
MadScience.py
README

README

----==== Mad Science ====----
WORK IN PROGRESS

Mad Science is a testing / benchmarking framework in python targeting server configuration benchmarking. 
Mad Science has this basic workflow: 
Restart services so you start from a known status, 
make one configuration change, 
restart services to apply the change, 
benchmark the server, 
record the results.  

Rinse, repeat.

A test (actually 200 tests in this case) looks like this:

# go through each value for thread_concurrency
for i in range(1, 21):
    replaceInConfig('thread_concurrency', 'thread_concurrency = %i' % (i), '/etc/my.cnf')
# run 10 tests at each level
    for x in range(1,11):
        testid = 'mysql-' + str(i) + '-' + str(x)
        doReset()
        runSiege('100','10','urls-local.txt')


Or maybe you want to test different SHM sizes in APC?

shmsizes = [8,16,24,32,48,64,72,96,128,160,192,256,384,512]

for i in shmsizes:
    replaceInConfig('apc.shm_size', 'apc.shm_size = %i' % (i), '/etc/php.d/apc.ini')
    for x in range(1,11):
        testid = 'apc-shm-' + str(i) + '-' + str(x)
        doReset()
        runSiege('100','100','urls-local.txt')



----==== Beaker ====----
WORK IN PROGRESS

Beaker is the statistics / graphing / analytical tool that works with Mad Science.  Right now it only parses Siege results based on your test selector.  Example that selects the "mysql-5*" tests:

[root@bench2 sandbox]# python26 Beaker.py mysql-5
Test ID selector: mysql-5
Raw dump of dataset to parse: 
{'hits': [1159, 1156, 0, 1172, 0, 1162, 1176, 510, 1179, 1170], 'short': [0.16, 0.22, 0.0, 0.25, 0.0, 0.19, 0.19, 0.41999999999999998, 0.20000000000000001, 0.19], 'concur': [92.129999999999995, 92.209999999999994, 0.0, 92.060000000000002, 0.0, 92.870000000000005, 91.609999999999999, 88.480000000000004, 91.629999999999995, 91.239999999999995], 'up': [100.0, 100.0, 0.0, 100.0, 0.0, 100.0, 100.0, 47.57, 100.0, 100.0], 'long': [14.92, 15.43, 0.0, 15.630000000000001, 0.0, 9.8300000000000001, 14.09, 15.41, 14.210000000000001, 15.34], 'resptime': [4.2999999999999998, 4.75, 0.0, 4.6500000000000004, 0.0, 3.3999999999999999, 4.5099999999999998, 4.8099999999999996, 4.5700000000000003, 4.4900000000000002], 'bw': [0.13, 0.12, 0.0, 0.12, 0.0, 0.16, 0.12, 0.10000000000000001, 0.12, 0.12], 'time': [54.07, 59.509999999999998, 341.94999999999999, 59.149999999999999, 344.94999999999999, 42.57, 57.93, 27.719999999999999, 58.780000000000001, 57.560000000000002], 'fail': [0, 0, 1000, 0, 1000, 0, 0, 562, 0, 0], 'ok': [1159, 1156, 0, 1172, 0, 1162, 1176, 515, 1179, 1170], 'trans': [21.440000000000001, 19.43, 0.0, 19.809999999999999, 0.0, 27.300000000000001, 20.300000000000001, 18.399999999999999, 20.059999999999999, 20.329999999999998], 'data': [7.0300000000000002, 6.9500000000000002, 0.0, 6.9900000000000002, 0.0, 6.9500000000000002, 7.0700000000000003, 2.73, 7.1399999999999997, 7.04]}

Mean: 
{'hits': 868.39999999999998, 'short': 0.182, 'concur': 73.222999999999999, 'up': 74.757000000000005, 'long': 11.486000000000001, 'resptime': 3.5479999999999996, 'bw': 0.099000000000000005, 'time': 110.41899999999998, 'fail': 256.19999999999999, 'ok': 868.89999999999998, 'trans': 16.707000000000001, 'data': 5.1899999999999995}

Median: 
{'hits': 1160.5, 'short': 0.19, 'concur': 91.620000000000005, 'up': 100.0, 'long': 14.565000000000001, 'resptime': 4.5, 'bw': 0.12, 'time': 58.355000000000004, 'fail': 0.0, 'ok': 1160.5, 'trans': 19.934999999999999, 'data': 6.9700000000000006}

Standard Deviation: 
{'hits': 501.58375837607286, 'short': 0.12016655108639841, 'concur': 38.609614821526861, 'up': 42.657309403612828, 'long': 6.2855304735028801, 'resptime': 1.9111706709065346, 'bw': 0.054252496102330017, 'time': 123.22752537481225, 'fail': 429.40004657661598, 'ok': 501.18913041330461, 'trans': 9.1290051179985898, 'data': 3.0460174362964865}

You can also use it to export data to CSV files.  Please see Beaker.py --help for the most up-to-date documentation.

 Usage: python26 Beaker.py [OPTION]... [test id selector]

    -h, --help
            This message (Usage)

    -f sqlitedb, --db=sqlitedb
            Parse [sqlitedb].  If none given, the default sqlitedb file name is './resultdb.sqlite'

    -e outputCSVFile, --csv=outputCSVFile
            Export siege results to CSV file.  Can be optionally used with a test ID selector to only export a subset of the results.

    "test id selector" should be a test id pattern to match against; 'apache-' would match 'apache-1' through 'apache-9999' and also 
'apache-vs-something-1

    If no test id selector is given, a list of all test IDs in the SQLite database will be shown.
    

Something went wrong with that request. Please try again.