Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Browse files

Small document describing the benchmark result pages and graphs.

  • Loading branch information...
commit b70fd8e461667564a3189c808eca9d1f507c2e6f 1 parent 1349e27
@schani schani authored
Showing with 65 additions and 0 deletions.
  1. +65 −0 RESULT-DOC.md
View
65 RESULT-DOC.md
@@ -0,0 +1,65 @@
+# Index
+
+The
+[index page](http://storage.bos.xamarin.com/mono-gcbench/index.html)
+shows all configurations on which the benchmark suite was run. For
+the last revision, it shows the average benchmark runtime relative to
+the best performing revision so far, as well as the relative runtime
+of the worst-performing benchmark of the current revision.
+
+The configuration overview graphs are explained below.
+
+# Configuration page
+
+[Configuration pages](http://storage.bos.xamarin.com/mono-gcbench/default-sgen/index.html)
+summarize individual configuration results. The table has a row for
+each benchmark, giving times and revisions for the best and worst
+results, as well as absolute and relative times for the latest three
+revisions.
+
+It also shows small versions of the benchmark graphs for each
+benchmark. Those graphs are explained below.
+
+In the rightmost column it gives graphs of native code size, as
+reported by `mono --stats`.
+
+## Configuration overview graph
+
+In the
+[configuration overview graph](http://storage.bos.xamarin.com/mono-gcbench/default-sgen/combined_large.png)
+revision time moves from left to right, i.e. earlier revisions are
+left, later revisions are right. The vertical axis is relative
+runtime, so lower is better.
+
+The main (black) line shows the average relative benchmark runtime,
+i.e. it gives a good measure of how each revision performs on
+average. The green and red lines show the best and worst performing
+individual benchmark runtimes per revision, i.e. they indicate whether
+a revision has any outliers in terms of performance.
+
+The green and red markers indicate the best and worst performing
+revisions, respectively, based on average relative benchmark runtime.
+
+# Individual benchmark page
+
+Each individual
+[benchmark page](http://storage.bos.xamarin.com/mono-gcbench/default-sgen/graph4.html)
+lists results for all revisions the particular benchmark was run on
+the particular configuration. It presents average, minimum and
+maximum run times (in seconds), native code size, and gives a link to
+the raw run time files.
+
+## Benchmark graph
+
+In the
+[benchmark graph](http://storage.bos.xamarin.com/mono-gcbench/default-sgen/graph4_large.png),
+just as in the configuration overview graph, revision time moves from
+left to right and the vertical axis indicates runtime, in this case
+absolute.
+
+The main (black) line shows the average runtime, over three runs,
+whereas the gray area shows the variation of the runs, i.e. slowest to
+fastest.
+
+The green and red markers indicate the best and worst performing
+revisions for the benchmark, respectively.
Please sign in to comment.
Something went wrong with that request. Please try again.