Skip to content

Commit

Permalink
Initial import
Browse files Browse the repository at this point in the history
  • Loading branch information
silentbicycle committed May 6, 2011
0 parents commit bf4fe2e
Show file tree
Hide file tree
Showing 7 changed files with 844 additions and 0 deletions.
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
ministat
ministat.cat1
ministat.o
14 changes: 14 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# $FreeBSD: src/usr.bin/ministat/Makefile,v 1.8 2008/10/16 20:39:02 phk Exp $
PROG= ministat
DPADD= ${LIBM}
LDADD= -lm

test: ${PROG}
./${PROG} < ${.CURDIR}/chameleon
./${PROG} ${.CURDIR}/chameleon
./${PROG} ${.CURDIR}/iguana ${.CURDIR}/chameleon
./${PROG} -c 80 ${.CURDIR}/iguana ${.CURDIR}/chameleon
./${PROG} -s -c 80 ${.CURDIR}/chameleon ${.CURDIR}/iguana
./${PROG} -s -c 80 ${.CURDIR}/chameleon ${.CURDIR}/iguana ${.CURDIR}/iguana

.include <bsd.prog.mk>
52 changes: 52 additions & 0 deletions README
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
PHK's ministat, adapted slightly for OpenBSD.

$FreeBSD: src/usr.bin/ministat/README,v 1.1.24.1.6.1 2010/12/21 17:09:25 kensmith Exp $

A small tool to do the statistics legwork on benchmarks etc.

Prepare your data into two files, one number per line
run
./ministat data_before data_after

and see what it says.

You need at least three data points in each data set, but the more
you have the better your result generally gets.

Here are two typical outputs:

x _1
+ _2
+--------------------------------------------------------------------------+
|x + x+ x x x + ++ |
| |_________|______AM_______________|__A___________M_______________||
+--------------------------------------------------------------------------+
N Min Max Median Avg Stddev
x 5 36060 36138 36107 36105.6 31.165686
+ 5 36084 36187 36163 36142.6 49.952978
No difference proven at 95.0% confidence

Here nothing can be concluded from the numbers. It _may_ be possible to
prove something if many more measurements are made, but with only five
measurements, nothing is proven.


x _1
+ _2
+--------------------------------------------------------------------------+
| + |
| x + +|
|x x x x + +|
| |_______________A_____M_________| |_M___A____| |
+--------------------------------------------------------------------------+
N Min Max Median Avg Stddev
x 5 0.133 0.137 0.136 0.1354 0.0015165751
+ 5 0.139 0.14 0.139 0.1394 0.00054772256
Difference at 95.0% confidence
0.004 +/- 0.00166288
2.95421% +/- 1.22812%
(Student's t, pooled s = 0.00114018)

Here we have a clearcut difference, not very big, but clear and unambiguous.


6 changes: 6 additions & 0 deletions chameleon
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# $FreeBSD: src/usr.bin/ministat/chameleon,v 1.1.24.1.6.1 2010/12/21 17:09:25 kensmith Exp $
150
400
720
500
930
8 changes: 8 additions & 0 deletions iguana
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# $FreeBSD: src/usr.bin/ministat/iguana,v 1.1.24.1.6.1 2010/12/21 17:09:25 kensmith Exp $
50
200
150
400
750
400
150
130 changes: 130 additions & 0 deletions ministat.1
Original file line number Diff line number Diff line change
@@ -0,0 +1,130 @@
.\"
.\" Copyright (c) 2007 Poul-Henning Kamp
.\" All rights reserved.
.\"
.\" Redistribution and use in source and binary forms, with or without
.\" modification, are permitted provided that the following conditions
.\" are met:
.\" 1. Redistributions of source code must retain the above copyright
.\" notice, this list of conditions and the following disclaimer.
.\" 2. Redistributions in binary form must reproduce the above copyright
.\" notice, this list of conditions and the following disclaimer in the
.\" documentation and/or other materials provided with the distribution.
.\"
.\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
.\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
.\" SUCH DAMAGE.
.\"
.\" $FreeBSD: src/usr.bin/ministat/ministat.1,v 1.2.2.1.6.1 2010/12/21 17:09:25 kensmith Exp $
.\"
.Dd December 20, 2007
.Dt MINISTAT 1
.Os
.Sh NAME
.Nm ministat
.Nd statistics utility
.Sh SYNOPSIS
.Nm
.Op Fl ns
.Op Fl C Ar column
.Op Fl c Ar confidence_level
.Op Fl d Ar delimiter
.Op Fl w Op width
.Op Ar
.Sh DESCRIPTION
The
.Nm
command calculates fundamental statistical properties of numeric data
in the specified files or, if no file is specified, standard input.
.Pp
The options are as follows:
.Bl -tag -width Fl
.It Fl n
Just report the raw statistics of the input, suppress the ASCII-art plot
and the relative comparisons.
.It Fl s
Print the average/median/stddev bars on separate lines in the ASCII-art
plot, to avoid overlap.
.It Fl C Ar column
Specify which column of data to use.
By default the first column in the input file(s) are used.
.It Fl c Ar confidence_level
Specify desired confidence level for Student's T analysis.
Possible values are 80, 90, 95, 98, 99 and 99.5 %
.It Fl d Ar delimiter
Specifies the column delimiter characters, default is SPACE and TAB.
See
.Xr strtok 3
for details.
.It Fl w Ar width
Width of ASCII-art plot in characters, default is 74.
.El
.Pp
A sample output could look like this:
.Bd -literal -offset indent
$ ministat -s -w 60 iguana chameleon
x iguana
+ chameleon
+------------------------------------------------------------+
|x * x * + + x +|
| |________M______A_______________| |
| |________________M__A___________________| |
+------------------------------------------------------------+
N Min Max Median Avg Stddev
x 7 50 750 200 300 238.04761
+ 5 150 930 500 540 299.08193
No difference proven at 95.0% confidence
.Ed
.Pp
If
.Nm
tells you, as in the example above, that there is no difference
proven at 95% confidence, the two data sets you gave it are for
all statistical purposes identical.
.Pp
You have the option of lowering your standards by specifying a
lower confidence level:
.Bd -literal -offset indent
$ ministat -c 80 iguana chameleon
x iguana
+ chameleon
+------------------------------------------------------------+
|x * x * + + x +|
| |________M______A_______________| |
| |________________M__A___________________| |
+------------------------------------------------------------+
N Min Max Median Avg Stddev
x 7 50 750 200 300 238.04761
+ 5 150 930 500 540 299.08193
Difference at 80.0% confidence
240 +/- 212.215
80% +/- 70.7384%
(Student's t, pooled s = 264.159)
.Ed
.Pp
But a lower standard does not make your data any better, and the
example is only included here to show the format of the output when
a statistical difference is proven according to Student's T method.
.Sh SEE ALSO
Any mathematics text on basic statistics, for instances Larry Gonicks
excellent "Cartoon Guide to Statistics" which supplied the above example.
.Sh HISTORY
The
.Nm
command was written by Poul-Henning Kamp out of frustration
over all the bogus benchmark claims made by people with no
understanding of the importance of uncertainty and statistics.
.Pp
From
.Fx 5.2
it has lived in the source tree as a developer tool, graduating
to the installed system from
.Fx 8.0 .
Loading

0 comments on commit bf4fe2e

Please sign in to comment.