Permalink
Browse files

Initial revision

  • Loading branch information...
bfulgham
bfulgham committed May 19, 2004
0 parents commit 26872eba22e77d5d875d77407e96fb09b8e76f83
Showing 973 changed files with 234,735 additions and 0 deletions.
No changes.
157 ChangeLog
@@ -0,0 +1,157 @@
+2000-12-03 (Doug Bagley)
+
+ * Switched from ChangeLog to automated Activity Log.
+
+ * Added some Pike programs from Fredrik Noring, et. al.
+
+2000-11-27 (Doug Bagley)
+
+ * Released version 0.1.7.
+
+2000-11-26 (Doug Bagley)
+
+ * Totally re-wrote the benchtest script, it is now called
+ minibench, and it has lots of nice features. It should start
+ getting easier for others to use it as a benchmarking tool. New
+ option --repeat should allow us to get a better idea of
+ variability in cputime from run to run.
+
+2000-11-23 (Doug Bagley)
+
+ * Released version 0.1.6 with latest fixes.
+
+ * Added a few more test programs in various areas.
+
+ * Added Reverse a File test.
+
+ * Dave Thomas offered some tips to speed up some of the Ruby
+ programs.
+
+2000-11-21 (Doug Bagley)
+
+ * Roberto Ierusalimschy has kindly provided more Lua test programs
+ for the new tests, in particular fixing problems in the Lua code
+ I've been attempting :-)
+
+2000-11-20 (Doug Bagley)
+
+ * Started new tests: lists, random number generator, heapsort.
+
+2000-11-19 (Doug Bagley)
+
+ * Released version 0.1.5 with latest fixes.
+
+ * I fixed a bug in the benchtest part of the framework that caused
+ CPU times from successive runs to be added. After re-running all
+ tests, a few of the rankings have changed a little. The most
+ noticable is that Java tests now show much better because they are
+ not penalized as much for their very high startup costs. Other
+ than that, the ranking orders didn't change too much.
+
+ * Roberto Ierusalimschy has kindly provided a number of the Lua
+ test programs.
+
+2000-11-06 (Doug Bagley)
+
+ * Modified benchtest to use the GTop perl module. Hopefully this
+ will be good for portability (at least to other Unices).
+
+2000-10-16 (Doug Bagley)
+
+ * Added a couple of Tcl wordfreqs.
+
+ * Added a bigloo sumcol test.
+
+2000-10-15 (Doug Bagley)
+
+ * Added Icon version of sumcol test.
+
+2000-10-12 (Doug Bagley)
+
+ * Added gcc moments test.
+
+ * Now you can "make report" at top level or in bench/ to see which
+ languages implement which tests, and which tests still need to be
+ written.
+
+ * Added a Tcl version of Ackermann, it explodes due to stack
+ overflow.
+
+2000-10-11 (Doug Bagley)
+
+ * Added Haskell version of Ackermann test.
+
+2000-10-09 (Doug Bagley)
+
+ * Added mercury sieve test.
+
+ * Added bigloo sieve test. Added -O3 param to bigloo compile.
+
+2000-10-08 (Doug Bagley)
+
+ * Added a couple SML test.
+
+ * Released version 0.1.3.
+
+ * Started Statistical Moments test.
+
+2000-10-07 (Doug Bagley)
+
+ * Removed some CGI stuff, and now we create HTML tables for data
+ at build time.
+
+ * Added an Ackermann test for bigloo scheme.
+
+2000-10-05 (Doug Bagley)
+
+ * Started sumcol test.
+
+ * The Ruby Ackermann test completes now (with the upgrade from
+ 1.4.6 to 1.6.1).
+
+ * Added Mercury version of Ackermann test.
+
+2000-10-03 (Doug Bagley)
+
+ * Released version 0.1.2.
+
+ * Added OCAML version of Ackermann and Sieve tests.
+
+2000-10-02 (Doug Bagley)
+
+ * Added Bill Lear's C++ spell checker that uses mmap.
+
+ * Started ChangeLog.
+
+2000-09-30 (Doug Bagley)
+
+ * Updated Ruby from 1.4.6 to 1.6.1.
+
+ * Updated Erlang from R7A to R7B.
+
+ * Updated Guile from 1.3.4 to 1.4.
+
+ * Updated Lua from 4.0alpha to 4.0beta.
+
+ * Updated Rep from 0.12.4 to 0.13.1.
+
+ * Updated gforth from 0.4 to 0.5.
+
+ * Updated gawk from 3.0.4 to 3.0.6.
+
+2000-09-25 (Doug Bagley)
+
+ * Now tests can define a range over which measurements are taken. The
+ range can be number of iterations, size of input datasets or a
+ parameter passed to the test so it can use it as an input parameter.
+
+ * The first test in the range can be designated as the one used for
+ correctness testing (comparing with expected output). It can also
+ serve to make sure input data is cached.
+
+ * Re-generated test data over variable ranges of input, and results
+ are now on the detail page for each test. At the minimum end we see
+ effects of startup costs. At the maximum end, we see the test's
+ optimal performance.
+
+
@@ -0,0 +1,47 @@
+# Language Benchmark Shootout Makefile Footer
+# $Id: Make.footer,v 1.1 2004-05-19 18:08:56 bfulgham Exp $
+
+# common rules go here
+
+.PHONY: show clean clobber setup loc post
+
+############################################################
+# Targets normally called by user
+############################################################
+all: plot data/loc.tab post data/all.html
+
+plot: setup
+ @$(RUNTESTS)
+
+test: setup
+ @$(RUNTESTS) --repeat 1
+
+show: plot
+ @ee data/max.png &
+ @ee data/cpu.png &
+ @ee data/mem.png &
+ @ee data/min.png &
+
+clobber: clean
+ @echo "Clobbering data/*"
+ @rm -rf data
+
+clean:
+ @echo "Cleaning tmp/*"
+ @rm -rf tmp
+
+############################################################
+# generate .loc in tmp/
+############################################################
+
+loc: data/loc.tab
+ cat $^
+
+tmp/%.loc: %
+ ../../bin/loc $(LOCDEPS) $^ > $@
+
+data/loc.tab: $(foreach prog,$(filter-out %~,$(wildcard $(TEST).*)),tmp/$(prog).loc)
+ cat $^ | sort > $@
+
+data/all.html: data/max.mbtab data/loc.tab
+ ../../bin/make_all_html
@@ -0,0 +1,51 @@
+# Language Benchmark Shootout Makefile Header
+# $Id: Make.header,v 1.1 2004-05-19 18:09:04 bfulgham Exp $
+
+############################################################
+# common definitions go here
+############################################################
+
+MINIBENCH := minibench --swapok #--debug
+
+BASH := /bin/bash
+BIGLOO := /usr/bin/bigloo
+CMUCL := /usr/bin/lisp
+ERLANG := /usr/bin/erl
+GAWK := /usr/bin/gawk
+GCC := /usr/bin/gcc
+GHC := /usr/bin/ghc
+GUILE := /usr/bin/guile
+GXX := /usr/bin/g++
+LUA := /usr/bin/lua
+MAWK := /usr/bin/mawk
+MLTON := /usr/bin/mlton
+MZSCHEME := /usr/bin/mzscheme
+OCAML := /usr/bin/ocamlopt
+OCAMLB := /usr/bin/ocamlc
+PERL := /usr/bin/perl
+PIKE := /usr/bin/pike
+PYTHON := /usr/bin/python
+RUBY := /usr/bin/ruby
+TCL := /usr/bin/tclsh
+XEMACS := /usr/bin/xemacs
+
+LANGS := $(BASH) $(BIGLOO) $(CMUCL) $(ERLANG) $(GAWK) \
+ $(GCC) $(GHC) $(GUILE) $(GXX) $(LUA) $(MAWK) \
+ $(MLTON) $(MZSCHEME) $(OCAML) $(OCAMLB) $(PERL) \
+ $(PIKE) $(PYTHON) $(RUBY) $(TCL) $(XEMACS)
+
+############################################################
+# we define a default rule to tell user what to do
+# if they just randomly type "make" here.
+############################################################
+default:
+ @echo "No default rule, try:"
+ @echo " make all # do almost everything"
+ @echo " make plot # create data and graphs"
+ @echo " make show # \"make plot\" and then display graphs"
+ @echo " make test # like \"make plot\", but run only one repetition of tests"
+ @echo " make clean # remove generated/data files to prep for full re-run"
+ @echo " make report # report which tests are written (or not)"
+
+.EXPORT_ALL_VARIABLES:
+
@@ -0,0 +1,34 @@
+# $Id: Makefile,v 1.1 2004-05-19 18:08:56 bfulgham Exp $
+
+include Make.header
+
+all: versions
+ (cd bench ; make $@)
+ (cd bench ; make --no-print-directory report> report.txt)
+ make recent craps codelinks
+
+test plot show loc:
+ (cd bench ; make $@)
+
+clean:
+ (cd bench ; make $@)
+
+.PHONY: dist report recent versions craps codelinks
+
+versions: versions.html
+
+versions.html: $(LANGS) bin/make_versions langs.pl
+ bin/make_versions > versions.html.tmp && \
+ mv -f versions.html.tmp versions.html
+
+dist:
+ bin/make_dist
+
+report codelinks:
+ @(cd bench ; make --no-print-directory $@)
+
+craps:
+ @(cd bench ; make --no-print-directory $@) > .craps.table
+
+recent:
+ @bin/make_recent > recent.html
@@ -0,0 +1,29 @@
+# $Id: Minibench.conf,v 1.1 2004-05-19 18:09:04 bfulgham Exp $
+#
+repeat 3
+rundir tmp
+datdir tmp
+tabdir data
+commandline cmucl: $CMUCL -noinit -batch -eval '(load "%B.cmucl_run" :verbose nil :print nil)' %A
+commandline gawk: $GAWK -f ../%X %A
+commandline ghc: %X %A $MB_GHCRTS
+commandline mawk: $MAWK -f ../%X %A
+commandline erlang: $ERLANG $ERLFLAGS -noinput -s %T main %A
+commandline lua: $LUA $LUA_EXTRAS ../%X %A
+commandline python: $PYTHON %B.pyo %A
+commandline tcl: $TCL ../%X %A
+commandline xemacs: $XEMACS -vanilla -batch -load %B.elc -funcall main %A
+automake gcc,bigloo,erlang,ghc
+automake ocaml,se,g++,xemacs,ocamlb,cmucl,python,mlton,njs
+maxtime 600
+minsummary 1
+maxsummary 1
+makedirs 1
+tabulate 1
+text 1
+graph 1
+verbose 1
+makefile ../../Makefile.mb
+update_file ../.up_date
+runlog tmp
+compareoutput 1
@@ -0,0 +1,18 @@
+
+Notes:
+- add new tests to:
+ - shootout/bench/Makefile
+ - shootout/craps.cgi
+ - shootout/index.shtml
+ - make symlinks for .ocaml -> ocamlb programs.
+- add new language:
+ - shortcuts/build.<lang>
+ - shootout/.htaccess (add extension as a text/plain type, if executable)
+ - shootout/Make.header
+ - shootout/Minibench.conf
+ - shootout/bench/hello/Minibench.conf
+ - shootout/bench/Makefile.mb (if needs to "compile")
+ - shootout/lang/<lang> (.cvsignore, index.shtml, allsrc.shtml)
+ - shootout/langs.pl
+ - bin/loc
+- do a "cvs update -d" after addition.
3 README
@@ -0,0 +1,3 @@
+To run the benchmarks, type "make"
+this will show you some fun targets to build.
+
7 Todo
@@ -0,0 +1,7 @@
+-*- mode: text -*-
+
+- languageshootout.org
+- db backend: postgresql
+- forums: propose test, comment on test, propose lang, comments
+ section for each test
+- remove Make dependancy and use only Perl
Oops, something went wrong.

0 comments on commit 26872eb

Please sign in to comment.