Skip to content
Browse files

Small document describing the benchmark result pages and graphs.

  • Loading branch information...
1 parent 1349e27 commit b70fd8e461667564a3189c808eca9d1f507c2e6f @schani schani committed Aug 16, 2012
Showing with 65 additions and 0 deletions.
  1. +65 −0
@@ -0,0 +1,65 @@
+# Index
+[index page](
+shows all configurations on which the benchmark suite was run. For
+the last revision, it shows the average benchmark runtime relative to
+the best performing revision so far, as well as the relative runtime
+of the worst-performing benchmark of the current revision.
+The configuration overview graphs are explained below.
+# Configuration page
+[Configuration pages](
+summarize individual configuration results. The table has a row for
+each benchmark, giving times and revisions for the best and worst
+results, as well as absolute and relative times for the latest three
+It also shows small versions of the benchmark graphs for each
+benchmark. Those graphs are explained below.
+In the rightmost column it gives graphs of native code size, as
+reported by `mono --stats`.
+## Configuration overview graph
+In the
+[configuration overview graph](
+revision time moves from left to right, i.e. earlier revisions are
+left, later revisions are right. The vertical axis is relative
+runtime, so lower is better.
+The main (black) line shows the average relative benchmark runtime,
+i.e. it gives a good measure of how each revision performs on
+average. The green and red lines show the best and worst performing
+individual benchmark runtimes per revision, i.e. they indicate whether
+a revision has any outliers in terms of performance.
+The green and red markers indicate the best and worst performing
+revisions, respectively, based on average relative benchmark runtime.
+# Individual benchmark page
+Each individual
+[benchmark page](
+lists results for all revisions the particular benchmark was run on
+the particular configuration. It presents average, minimum and
+maximum run times (in seconds), native code size, and gives a link to
+the raw run time files.
+## Benchmark graph
+In the
+[benchmark graph](,
+just as in the configuration overview graph, revision time moves from
+left to right and the vertical axis indicates runtime, in this case
+The main (black) line shows the average runtime, over three runs,
+whereas the gray area shows the variation of the runs, i.e. slowest to
+The green and red markers indicate the best and worst performing
+revisions for the benchmark, respectively.

0 comments on commit b70fd8e

Please sign in to comment.
Something went wrong with that request. Please try again.