This contains notes on running benchmarks on these binaries. Generally this is test the speed of QEMU and it’s emulation code.
Binaries To Test
Here we have several tables of binaries to test. We refer to the current benchmarking set from the next stage, Run Benchmark.
For a final test we might compare the system QEMU with a reference build as well as our current build.
We might want to copy QEMU across at the start of each patch we re-factor so we can cleanly test without getting lost in the changes of a large series.
In fact while developing we can just keep running against the current build and comparing against previously recorded logs.
This runs the benchmark against each binary we have selected above.
import subprocess import os runs= for qemu,logname in files: cmd="taskset -c 0 %s ./vector-benchmark -n %s | tee %s" % (qemu, tests, logname) subprocess.call(cmd, shell=True) runs.append(logname) return runs
Before we graph the results we need to import the data from the run into a table of data we can now feed into gnuplot.
For later processing we need a simple count of the number of tests we actually ran. We could calculate this from the variable tests in Run Benchmark but it is just as simple to count the lines of output.
cat $data | wc -l
This fragment munges the run data (which we did asynchronously) into the org-mode table from which we can then use to feed gnuplot.
nruns = len(data) table = [[0 for x in range(nruns+1)] for y in range(ntests+1)] table = "test/run" for logfile, col in zip(data, xrange(1, nruns+1)): table[col] = "%s" % (logfile.rstrip(".log")) with open(logfile) as f: row = xrange(1, ntests+1) for row, line in enumerate(f, 1): (test,result) = line.split(",") table[row] = test table[row][col] = result.rstrip() return table
Finally with the results nicely tabulated we can call GNUPlot with the data to plot our nice bar charts.
set title "Vector Benchmark Results (lower is better)" set style data histograms set style fill solid 1.0 border lt -1 set xtics rotate by 90 right set yrange [:] set xlabel noenhanced set ylabel "nsecs/Kop" noenhanced set xtics noenhanced set ytics noenhanced set boxwidth 1 set xtics format "" set xtics scale 0 set grid ytics set term pngcairo size 1200,500 plot for [i=2:5] data using i:xtic(1) title columnhead
This is a simple restful script to upload to imgur. While restful-mode is intended for live checking of REST APIs it can easily be embedded in an org-babel source block.
:client_id = fd2da649643c743 # Upload images to imgur POST https://api.imgur.com/3/image Authorization: Client-ID :client_id Content-type: image/png < benchmark.png
This is a simple helper function to call the above code but strip out the URL so you don’t need to hunt through all the headers.
(when (string-match (rx "link" (one-or-more (any "\":" whitespace)) (group (one-or-more (not (any "\""))))) json-string) (match-string 1 json-string))