FAQ

Why does it take Fibon so long to run?

I can think of 5 ways to lower the overall run time:

1. Run with only one tune setting
2. Run fewer iterations
3. Limit benchmark execution time
4. Run fewer benchmarks
5. Run with smaller inputs

Some of these changes can be made in the config file or from the command line. See the fibon-config github repo for some example config files.

Run with only one tune setting

I partly modeled Fibon after the SPEC benchmarks which have a notion of different "tune" settings: base and peak. The default config has base as -O0 and peak as -O2. Having two tune settings means that every benchmark will be compiled twice and run at least twice. You can run only one tune setting by setting it in the config file:

tuneList = [Peak]

or on the command line:

$ fibon-run --tune=Peak

Run fewer iterations

You can run each benchmark fewer times by specifying the iterations in the config file

iterations = 3

or on the command line

$ fibon-run --iters=3

Limit benchmark execution time

You can limit the execution time for the benchmarks. This can be an easy way to keep a benchmark from eating up all the execution time for the suite. You can set this in the config file. To limit execution time for all benchmarks, put it in the default builder section:

build :: ConfigBuilder
build ConfigTuneDefault ConfigBenchDefault = do
  setTimeout $ Limit 0 0 15

The constructor Limit takes ints for the maximum number of hours, minutes, and seconds. So Limit 0 0 15 will limit all benchmark runs to 15 seconds. You can also set this on a per-benchmark or per-tune basis:

build (ConfigTuneDefault) (ConfigBench Cpsa) = do
  -- give Cpsa a little longer
  setTimeout $ Limit 0 0 30

This currently only works to limit the actual running time of a benchmark, not the compile time.

Run fewer benchmarks

You can run a subset of the full suite. Cutting out the benchmarks that take a long time to compile or run can make it complete much faster. You can set the run list in the configuration file

runList  =  [RunSingle agum, RunGroup Repa]

or from the command line

$ fibon-run Agum Repa

Run with smaller inputs

A final option would be to run with smaller inputs. Just like the SPEC benchmarks, the Fibon suite has both a "test" and a "ref" size. The "test" size is not likely to give a very accurate performance difference, but it runs very quickly so it can be useful for testing. You can set it from the config file:

sizeList = [Test]

or on the command line: $ fibon-run --size=Test

A more labor intense way would be to change the actual arguments used for the "ref" size. For the benchmarks that just take a command line parameter this can be easily set from the config file. For example to pass 1000 as the parameter to the QSort benchmark:

build (ConfigTune Peak) (ConfigBench QSort) = do
  replace RunFlags "1000 +RTS -tghc.stats --machine-readable -RTS"

Depending on the benchmark this may cause some validation failure if the output does not match the expected output.