This repository contains the benchmark suite and the harness used to evaluate the Stopify compiler.
The benchmarks are hosted in the
stopify-aec repository. For
each language, the original benchmarks are stored within the
benchmark is stored in
The Stopify benchmarking harness is contianed within
./harness/ folder. This
is the entrypoint to the benchmarking harness. The harness is responsible for
- Initializing the SQLite database that contains all the evaluation data.
- Compiling the benchmark files with the Stopify compiler.
- Running the server to collect benchmarking information from each browser.
Building the harness
NOTE: Make sure that
./harness/results.sqlite is removed befre running
To build the harness, run the following commands.
> cd harness/ > rm results.sqlite # Remove the checked-in version of evaluation numbers. > yarn install && yarn run build # Build the harness > yarn run init # Initialize the DB. > yarn run compileJs # Compile all benchmarks with Stopify
The benchmarks compiled by the harness are placed in
Running the benchmarks
Run the following command from
./harness after building the harness.
> yarn run server
Once the harness server is running, visit
localhost:4997 in the browser you
want to benchmark. The benchmarks will automatically start running. Since there
are a lot of benchmarks, it will take quite a while to run all the benchmarks.
Once the benchmarks are done running, the webpage will allow you to render
the results by clicking on "Render results". Clicking on this will generate
graphs for the run and also save these in the