From ea12e97c857c5a5e172df5a87637c9789372d2de Mon Sep 17 00:00:00 2001 From: Bryan O'Sullivan Date: Thu, 1 Dec 2011 22:03:27 -0800 Subject: [PATCH] Add the tiny.html example report. --- criterion.cabal | 10 +- examples/tiny.html | 668 +++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 672 insertions(+), 6 deletions(-) create mode 100644 examples/tiny.html diff --git a/criterion.cabal b/criterion.cabal index f901a5b3..6ca1e6d0 100644 --- a/criterion.cabal +++ b/criterion.cabal @@ -14,14 +14,12 @@ cabal-version: >= 1.6 extra-source-files: README.markdown examples/*.hs + examples/*.html data-files: - templates/js/excanvas-r3.min.js - templates/js/jquery-1.6.4.min.js - templates/js/jquery.criterion.js - templates/js/jquery.flot-0.7.min.js - templates/criterion.css - templates/report.tpl + templates/js/*.js + templates/*.css + templates/*.tpl description: This library provides a powerful but simple way to measure software diff --git a/examples/tiny.html b/examples/tiny.html new file mode 100644 index 00000000..9c7237ee --- /dev/null +++ b/examples/tiny.html @@ -0,0 +1,668 @@ + + + + + criterion report + + + + + + + + +
+
+

criterion performance measurements

+ +

overview

+ +

want to understand this report?

+ +
+ +

fib/fib 10

+ + + + + + + +
+ + + + + + + + + + + + + + + + + + + + + +
lower boundestimateupper bound
Mean execution time7.950250148027324e-78.087233523139602e-78.263881789521396e-7
Standard deviation6.469576002518438e-87.962093335887086e-89.595188287095253e-8
+ + +

Outlying measurements have severe + (0.789683189845111%) + effect on estimated standard deviation.

+ +

fib/fib 20

+ + + + + + + +
+ + + + + + + + + + + + + + + + + + + + + +
lower boundestimateupper bound
Mean execution time9.397387721886237e-59.511279240520537e-59.683396722581508e-5
Standard deviation5.084981483933464e-67.060410460215048e-69.916444086871226e-6
+ + +

Outlying measurements have severe + (0.6764999252677036%) + effect on estimated standard deviation.

+ +

fib/fib 30

+ + + + + + + +
+ + + + + + + + + + + + + + + + + + + + + +
lower boundestimateupper bound
Mean execution time1.1405421545418602e-21.1464177420052388e-21.166933422318349e-2
Standard deviation1.5835878091381052e-45.030517750313856e-41.146763021342376e-3
+ + +

Outlying measurements have moderate + (0.414995643896579%) + effect on estimated standard deviation.

+ +

intmap/intmap 25k

+ + + + + + + +
+ + + + + + + + + + + + + + + + + + + + + +
lower boundestimateupper bound
Mean execution time5.030530741127831e-35.067442705544335e-35.11489753952871e-3
Standard deviation1.7601420712288937e-42.14104721044797e-42.796949297562274e-4
+ + +

Outlying measurements have moderate + (0.39528660323872544%) + effect on estimated standard deviation.

+ +

intmap/intmap 50k

+ + + + + + + +
+ + + + + + + + + + + + + + + + + + + + + +
lower boundestimateupper bound
Mean execution time1.3004106333168846e-21.3085197260292869e-21.317617540589223e-2
Standard deviation3.817067615429757e-44.4020726935288003e-45.281243811580562e-4
+ + +

Outlying measurements have moderate + (0.2967324863902443%) + effect on estimated standard deviation.

+ +

intmap/intmap 75k

+ + + + + + + +
+ + + + + + + + + + + + + + + + + + + + + +
lower boundestimateupper bound
Mean execution time1.772755031815419e-21.782442693940053e-21.794612770310293e-2
Standard deviation4.572927417104507e-45.554628346393567e-47.805157733235236e-4
+ + +

Outlying measurements have moderate + (0.2673858721467461%) + effect on estimated standard deviation.

+ + +

understanding this report

+ +

In this report, each function benchmarked by criterion is assigned + a section of its own. In each section, we display two charts, each + with an x axis that represents measured execution time. + These charts are active; if you hover your mouse over data points + and annotations, you will see more details.

+ +
    +
  • The chart on the left is a + kernel + density estimate (also known as a KDE) of time + measurements. This graphs the probability of any given time + measurement occurring. A spike indicates that a measurement of a + particular time occurred; its height indicates how often that + measurement was repeated.
    • + +
  • The chart on the right is the raw data from which the kernel + density estimate is built. Measurements are displayed on + the y axis in the order in which they occurred.
    • +
+ +

Under the charts is a small table displaying the mean and standard + deviation of the measurements. We use a statistical technique + called + the bootstrap + to provide confidence intervals on our estimates of these values. + The bootstrap-derived upper and lower bounds on the mean and + standard deviation let you see how accurate we believe those + estimates to be. (Hover the mouse over the table headers to see + the confidence levels.)

+ +

A noisy benchmarking environment can cause some or many + measurements to fall far from the mean. These outlying + measurements can have a significant inflationary effect on the + estimate of the standard deviation. We calculate and display an + estimate of the extent to which the standard deviation has been + inflated by outliers.

+ + + +
+
+ + +