Benchmarks for various Scheme implementations. Taken with kind permission from the Larceny project, based on the Gabriel and Gambit benchmarks.
Scheme HTML TeX JavaScript Shell KiCad
Switch branches/tags
Nothing to show
Latest commit 031e847 Jun 9, 2017 @ecraven new sequential run
Permalink
Failed to load latest commit information.
inputs update pi.scm, add chudnovsky.scm. fixes #20 Jun 7, 2017
outputs Make sure ./outputs exists May 4, 2016
src update pi.scm, add chudnovsky.scm. fixes #20 Jun 7, 2017
.gitignore Do not ignore results.* and *.csv any more May 31, 2016
Makefile new run Jun 7, 2017
Makefile.schemes new run Jun 7, 2017
README.org Add Cyclone Scheme to readme (#19) Aug 29, 2016
all.csv new sequential run Jun 9, 2017
bench update pi.scm, add chudnovsky.scm. fixes #20 Jun 7, 2017
benchmark.html new sequential run Jun 9, 2017
results.Bigloo new sequential run Jun 9, 2017
results.Bones new sequential run Jun 9, 2017
results.Chez new sequential run Jun 9, 2017
results.Chibi new sequential run Jun 9, 2017
results.Chicken new sequential run Jun 9, 2017
results.ChickenCSI new sequential run Jun 9, 2017
results.Cyclone new sequential run Jun 9, 2017
results.Femtolisp new sequential run Jun 9, 2017
results.Foment new sequential run Jun 9, 2017
results.GambitC new sequential run Jun 9, 2017
results.Gauche new sequential run Jun 9, 2017
results.Guile new sequential run Jun 9, 2017
results.IronScheme new sequential run Jun 9, 2017
results.Kawa new sequential run Jun 9, 2017
results.Larceny new sequential run Jun 9, 2017
results.MIT new sequential run Jun 9, 2017
results.Mosh new sequential run Jun 9, 2017
results.Petite new sequential run Jun 9, 2017
results.Picrin new sequential run Jun 9, 2017
results.RScheme new sequential run Jun 9, 2017
results.Racket new sequential run Jun 9, 2017
results.Rhizome new sequential run Jun 9, 2017
results.S9fES new sequential run Jun 9, 2017
results.Sagittarius new sequential run Jun 9, 2017
results.Scheme48 new sequential run Jun 9, 2017
results.Stalin new sequential run Jun 9, 2017
results.TinyScheme new sequential run Jun 9, 2017
results.Vicare new sequential run Jun 9, 2017
results.Ypsilon new sequential run Jun 9, 2017
summarize.sch Commit based on Larceny May 4, 2016
summarize2.sch Commit based on Larceny May 4, 2016

README.org

R7RS Benchmarks

Benchmarks for various Scheme implementations. Taken with kind permission from the Larceny project, based on the Gabriel and Gambit benchmarks.

See the results here

Running the tests

Just run

./bench <scheme> <test>

Schemes that should work:

This should result in a file ./results.<scheme>.

All of these Schemes are packaged for ArchLinux. The tests are run with whatever that package produces.

Run

make csv

to produce a file all.csv with the test results (<scheme>,<test>,<seconds>).

Limiting compile-time and run-time

The environment variable CPU_TIME can be set, to limit the time for compilation and for running (each individually).

CPU_TIME=3600 ./bench chez all

Path to executables

You can set an environment variable to specify the path to each Scheme.

LARCENY=/tmp/larceny/larceny MIT=/tmp/mit/mit-scheme ./bench "mit larceny" "fib ack"

Notes for specific implementations

Chicken

The chicken test code assumes the presence of

  • vector-lib
  • r7rs

Install them by running

chicken-install vector-lib r7rs

Racket

The racket test code assumes the presence of https://github.com/lexi-lambda/racket-r7rs.git.

Install the r7rs package installation-wide.

Run as root:

raco pkg install -i r7rs r7rs-lib

IronScheme

Precompile system libraries (optional). It is still going to be slow.

echo "(compile-system-libraries)" | mono IronScheme.Console-v4.exe

The default binary for IronScheme (can be changed by exporting IRONSCHEME in your shell before running bench) is called ironscheme.

If you unpack IronScheme in your home, you can create a shell script like the following:

#!/bin/bash
mono ~/IronScheme/IronScheme.Console-v4.exe -nologo -emacs $*

and put that somewhere in your $PATH.

Stalin

Before calling Stalin, the code is run through alexpander (by default using Chicken’s csi). Then stalin is invoked on the result.

Unsupported Schemes

Owl Lisp

Missing a lot of necessary functions, read for example.

TinyScheme

Problems with redirecting input, read chokes. No timing functions.