Skip to content


Repository files navigation

The Eta Benchmark Suite

Build Status

This repository contains a set of performance tests used to properly measure how changes in the compiler affect performance. A big part of this suite consists of the nofib benchmark suite along with benchmarks that are relevant for Eta.


The Java Micobenchmarking Harness is used to conduct the benchmarks. The follow process occurs when you run a benchmark:

  1. The benchmark is compiled into a JAR file.
  2. The eta.main.main method (the entry point into Eta programs) is invoked from the JMH framework.
  3. The JMH framework will fork a single JVM process and run the program a number of times for warmup and a number of times for measurement and the average/error is calculated. At the end of each iteration, GC is run and all static thunks are reset to unevaluated state.

Getting Started


  • Gradle
  • JDK 1.7+
  • etlas

Runner Installation

First, get setup:

$ ./scripts/

Quick Run

A standalone script will run the fast part of the suite, also run by CircleCI.

$ ./scripts/

Slow Run

A standalone script will run the slow part of the suite.

$ ./scripts/

Run a Single Benchmark

A single benchmark can be run:

./scripts/ [test-name] [measurement-iterations] [warmup-iterations]


./scripts/ imaginary/bernouilli 10 5

This indicates 10 measurement iterations and 5 warmup iterations.

Note: You can also pass --skip-check as a parameter to skip the checking of the test output.

If you want to do more fine-grained performance benchmarking, please see the remaining sections.


All runs of the runner should be at the root of the repository.

To run an individual test

$ eta-bench [test-name] --run


$ eta-bench scs --run

To run an entire suite

$ eta-bench [suite-name] --run


$ eta-bench imaginary --run

Clean up test/benchmark artifacts

$ eta-bench clean

Contact Us

If you had trouble using this project, you can give us feedback by: