Skip to content

Replace criterion with gauge as the benchmark framework#571

Merged
treeowl merged 1 commit intohaskell:masterfrom
sjakobi:gauge
Apr 10, 2019
Merged

Replace criterion with gauge as the benchmark framework#571
treeowl merged 1 commit intohaskell:masterfrom
sjakobi:gauge

Conversation

@sjakobi
Copy link
Copy Markdown
Member

@sjakobi sjakobi commented Oct 13, 2018

This removes the circular dependency on criterion containers in the benchmarks.

One consequence of this is that we can now simply use cabal new-bench
to run the benchmarks.

@sjakobi
Copy link
Copy Markdown
Member Author

sjakobi commented Oct 13, 2018

For some previous discussion on using gauge, see #462 (comment) and the following comments.

gauge has developed quite a bit since. See http://hackage.haskell.org/package/gauge/changelog.

@ethercrow
Copy link
Copy Markdown
Contributor

bump @treeowl

This circular dependency is really a very annoying barrier of entry to working on containers and unordered-containers.

@sjakobi
Copy link
Copy Markdown
Member Author

sjakobi commented Mar 17, 2019

Bump @treeowl.

At this point, gauge offers a very similar feature set as criterion:

≻ cabal run -- sequence-benchmarks -h
Up to date
Microbenchmark suite - built with gauge 0.2.4
  -I CI     --ci=CI                   Confidence interval
  -G        --no-gc                   Do not collect garbage between iterations
  -L SECS   --time-limit=SECS         Min seconds for each benchmark run, default is 5.0 in normal mode, 0.0 in quick mode
            --min-samples=COUNT       Min no. of samples for each benchmark, default is 10 in normal mode, 1 in quick mode
            --min-duration=MILLISECS  Min duration for each sample, default is MilliSeconds 30, when 0 stops after first iteration
            --include-first-iter      Do not discard the measurement of the first iteration
  -q        --quick                   Perform a quick measurement and report results without statistical analysis
            --measure-only=FILE       Just measure the benchmark and place the raw data in the given file
            --measure-with=FILE       Perform measurements in a separate process using this program.
            --resamples=COUNT         Number of boostrap resamples to perform
            --regress=RESP:PRED..     Regressions to perform
            --raw=FILE                File to write raw data to
  -o FILE   --output=FILE             File to write report to
            --csvraw=FILE             File to write CSV measurements to
            --csv=FILE                File to write CSV summary to
            --json=FILE               File to write JSON summary to
            --junit=FILE              File to write JUnit summary to
  -v LEVEL  --verbosity=LEVEL         Verbosity level
  -t FILE   --template=FILE           Template to use for report
  -n ITERS  --iters=ITERS             Run benchmarks, don't analyse
  -m MATCH  --match=MATCH             Benchmark match style: prefix (default), exact, pattern (substring), or ipattern (case insensitive)
  -l        --list                    List benchmarks
            --version                 Show version info
  -s        --small                   Set benchmark display to the minimum useful information
  -h        --help                    Show help

@sjakobi
Copy link
Copy Markdown
Member Author

sjakobi commented Apr 10, 2019

Bump @treeowl!

How about you just merge this and give it a try? If it turns out to be a bad idea, the change should be easily reverted!

The reduced barrier for new contributors should make this easily worth it.

@treeowl
Copy link
Copy Markdown
Contributor

treeowl commented Apr 10, 2019

Okay!

@treeowl treeowl merged commit 8f42a47 into haskell:master Apr 10, 2019
@sjakobi sjakobi deleted the gauge branch April 10, 2019 20:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants