-
Notifications
You must be signed in to change notification settings - Fork 1
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
0 parents
commit bf4fe2e
Showing
7 changed files
with
844 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,3 @@ | ||
ministat | ||
ministat.cat1 | ||
ministat.o |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,14 @@ | ||
# $FreeBSD: src/usr.bin/ministat/Makefile,v 1.8 2008/10/16 20:39:02 phk Exp $ | ||
PROG= ministat | ||
DPADD= ${LIBM} | ||
LDADD= -lm | ||
|
||
test: ${PROG} | ||
./${PROG} < ${.CURDIR}/chameleon | ||
./${PROG} ${.CURDIR}/chameleon | ||
./${PROG} ${.CURDIR}/iguana ${.CURDIR}/chameleon | ||
./${PROG} -c 80 ${.CURDIR}/iguana ${.CURDIR}/chameleon | ||
./${PROG} -s -c 80 ${.CURDIR}/chameleon ${.CURDIR}/iguana | ||
./${PROG} -s -c 80 ${.CURDIR}/chameleon ${.CURDIR}/iguana ${.CURDIR}/iguana | ||
|
||
.include <bsd.prog.mk> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,52 @@ | ||
PHK's ministat, adapted slightly for OpenBSD. | ||
|
||
$FreeBSD: src/usr.bin/ministat/README,v 1.1.24.1.6.1 2010/12/21 17:09:25 kensmith Exp $ | ||
|
||
A small tool to do the statistics legwork on benchmarks etc. | ||
|
||
Prepare your data into two files, one number per line | ||
run | ||
./ministat data_before data_after | ||
|
||
and see what it says. | ||
|
||
You need at least three data points in each data set, but the more | ||
you have the better your result generally gets. | ||
|
||
Here are two typical outputs: | ||
|
||
x _1 | ||
+ _2 | ||
+--------------------------------------------------------------------------+ | ||
|x + x+ x x x + ++ | | ||
| |_________|______AM_______________|__A___________M_______________|| | ||
+--------------------------------------------------------------------------+ | ||
N Min Max Median Avg Stddev | ||
x 5 36060 36138 36107 36105.6 31.165686 | ||
+ 5 36084 36187 36163 36142.6 49.952978 | ||
No difference proven at 95.0% confidence | ||
|
||
Here nothing can be concluded from the numbers. It _may_ be possible to | ||
prove something if many more measurements are made, but with only five | ||
measurements, nothing is proven. | ||
|
||
|
||
x _1 | ||
+ _2 | ||
+--------------------------------------------------------------------------+ | ||
| + | | ||
| x + +| | ||
|x x x x + +| | ||
| |_______________A_____M_________| |_M___A____| | | ||
+--------------------------------------------------------------------------+ | ||
N Min Max Median Avg Stddev | ||
x 5 0.133 0.137 0.136 0.1354 0.0015165751 | ||
+ 5 0.139 0.14 0.139 0.1394 0.00054772256 | ||
Difference at 95.0% confidence | ||
0.004 +/- 0.00166288 | ||
2.95421% +/- 1.22812% | ||
(Student's t, pooled s = 0.00114018) | ||
|
||
Here we have a clearcut difference, not very big, but clear and unambiguous. | ||
|
||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,6 @@ | ||
# $FreeBSD: src/usr.bin/ministat/chameleon,v 1.1.24.1.6.1 2010/12/21 17:09:25 kensmith Exp $ | ||
150 | ||
400 | ||
720 | ||
500 | ||
930 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,8 @@ | ||
# $FreeBSD: src/usr.bin/ministat/iguana,v 1.1.24.1.6.1 2010/12/21 17:09:25 kensmith Exp $ | ||
50 | ||
200 | ||
150 | ||
400 | ||
750 | ||
400 | ||
150 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,130 @@ | ||
.\" | ||
.\" Copyright (c) 2007 Poul-Henning Kamp | ||
.\" All rights reserved. | ||
.\" | ||
.\" Redistribution and use in source and binary forms, with or without | ||
.\" modification, are permitted provided that the following conditions | ||
.\" are met: | ||
.\" 1. Redistributions of source code must retain the above copyright | ||
.\" notice, this list of conditions and the following disclaimer. | ||
.\" 2. Redistributions in binary form must reproduce the above copyright | ||
.\" notice, this list of conditions and the following disclaimer in the | ||
.\" documentation and/or other materials provided with the distribution. | ||
.\" | ||
.\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND | ||
.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE | ||
.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE | ||
.\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE | ||
.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL | ||
.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS | ||
.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) | ||
.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT | ||
.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY | ||
.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF | ||
.\" SUCH DAMAGE. | ||
.\" | ||
.\" $FreeBSD: src/usr.bin/ministat/ministat.1,v 1.2.2.1.6.1 2010/12/21 17:09:25 kensmith Exp $ | ||
.\" | ||
.Dd December 20, 2007 | ||
.Dt MINISTAT 1 | ||
.Os | ||
.Sh NAME | ||
.Nm ministat | ||
.Nd statistics utility | ||
.Sh SYNOPSIS | ||
.Nm | ||
.Op Fl ns | ||
.Op Fl C Ar column | ||
.Op Fl c Ar confidence_level | ||
.Op Fl d Ar delimiter | ||
.Op Fl w Op width | ||
.Op Ar | ||
.Sh DESCRIPTION | ||
The | ||
.Nm | ||
command calculates fundamental statistical properties of numeric data | ||
in the specified files or, if no file is specified, standard input. | ||
.Pp | ||
The options are as follows: | ||
.Bl -tag -width Fl | ||
.It Fl n | ||
Just report the raw statistics of the input, suppress the ASCII-art plot | ||
and the relative comparisons. | ||
.It Fl s | ||
Print the average/median/stddev bars on separate lines in the ASCII-art | ||
plot, to avoid overlap. | ||
.It Fl C Ar column | ||
Specify which column of data to use. | ||
By default the first column in the input file(s) are used. | ||
.It Fl c Ar confidence_level | ||
Specify desired confidence level for Student's T analysis. | ||
Possible values are 80, 90, 95, 98, 99 and 99.5 % | ||
.It Fl d Ar delimiter | ||
Specifies the column delimiter characters, default is SPACE and TAB. | ||
See | ||
.Xr strtok 3 | ||
for details. | ||
.It Fl w Ar width | ||
Width of ASCII-art plot in characters, default is 74. | ||
.El | ||
.Pp | ||
A sample output could look like this: | ||
.Bd -literal -offset indent | ||
$ ministat -s -w 60 iguana chameleon | ||
x iguana | ||
+ chameleon | ||
+------------------------------------------------------------+ | ||
|x * x * + + x +| | ||
| |________M______A_______________| | | ||
| |________________M__A___________________| | | ||
+------------------------------------------------------------+ | ||
N Min Max Median Avg Stddev | ||
x 7 50 750 200 300 238.04761 | ||
+ 5 150 930 500 540 299.08193 | ||
No difference proven at 95.0% confidence | ||
.Ed | ||
.Pp | ||
If | ||
.Nm | ||
tells you, as in the example above, that there is no difference | ||
proven at 95% confidence, the two data sets you gave it are for | ||
all statistical purposes identical. | ||
.Pp | ||
You have the option of lowering your standards by specifying a | ||
lower confidence level: | ||
.Bd -literal -offset indent | ||
$ ministat -c 80 iguana chameleon | ||
x iguana | ||
+ chameleon | ||
+------------------------------------------------------------+ | ||
|x * x * + + x +| | ||
| |________M______A_______________| | | ||
| |________________M__A___________________| | | ||
+------------------------------------------------------------+ | ||
N Min Max Median Avg Stddev | ||
x 7 50 750 200 300 238.04761 | ||
+ 5 150 930 500 540 299.08193 | ||
Difference at 80.0% confidence | ||
240 +/- 212.215 | ||
80% +/- 70.7384% | ||
(Student's t, pooled s = 264.159) | ||
.Ed | ||
.Pp | ||
But a lower standard does not make your data any better, and the | ||
example is only included here to show the format of the output when | ||
a statistical difference is proven according to Student's T method. | ||
.Sh SEE ALSO | ||
Any mathematics text on basic statistics, for instances Larry Gonicks | ||
excellent "Cartoon Guide to Statistics" which supplied the above example. | ||
.Sh HISTORY | ||
The | ||
.Nm | ||
command was written by Poul-Henning Kamp out of frustration | ||
over all the bogus benchmark claims made by people with no | ||
understanding of the importance of uncertainty and statistics. | ||
.Pp | ||
From | ||
.Fx 5.2 | ||
it has lived in the source tree as a developer tool, graduating | ||
to the installed system from | ||
.Fx 8.0 . |
Oops, something went wrong.