Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
3 changed files
with
75 additions
and
32 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,21 +1,66 @@ | ||
# Comparing benchmarks for regression | ||
|
||
Build and run the benchmark executable with `--csvraw=results.csv` option for | ||
the baseline code i.e. without the changes. | ||
The following commands are tested with `cabal` version 3.0. | ||
|
||
Run the benchmarks for the baseline code i.e. without the changes: | ||
|
||
``` | ||
$ cabal v2-run bench -- --csvraw=results.csv unicode-transforms | ||
# Remove any old benchmark results file first | ||
$ rm results.csv | ||
$ cabal run bench -- --csvraw=results.csv --quick | ||
``` | ||
|
||
It will collect the benchmark results in `results.csv` file. | ||
|
||
Now repeat the above step for the new code i.e. with the changes. | ||
It will append the benchmarks results to the existing csv file. | ||
It will append the new benchmarks results to the existing `results.csv` file. | ||
|
||
If you want the benchmarks to run quickly for a quick overview then use | ||
`--quick` option when running the benchmarking executable. | ||
If you want more accurate benchmark results remove the `--quick` flag. | ||
|
||
Build the bench-show reporting executable `chart`. | ||
To generate a benchmark comparison between old and new changes from the | ||
benchmark results in `results.csv` file: | ||
|
||
``` | ||
# [NOTE] The path "results.csv" is harcoded in the chart executable. | ||
$ cabal v2-run chart | ||
$ cabal run chart --flag bench-show | ||
``` | ||
|
||
# Comparing with ICU (text-icu package) | ||
|
||
Install the icu library: | ||
|
||
``` | ||
# On Mac OS using MacPorts | ||
$ sudo port install icu | ||
# On Mac OS using brew | ||
$ brew install icu4c | ||
# On Debian Linux based distributions (the library version suffix may differ) | ||
$ sudo apt-get install libicu65 | ||
``` | ||
|
||
If cabal cannot automatically find the icu library (e.g. when installed | ||
via macports install) then use explicit `LIBRARY_PATH` to tell it where | ||
the library is: | ||
|
||
``` | ||
$ export LIBRARY_PATH=/usr/lib/:opt/local/lib | ||
# Alternatively, pass the lib and include path as follows | ||
$ cabal bench --extra-lib-dirs=/usr/local/opt/icu4c/lib --extra-include-dirs=/usr/local/opt/icu4c/include | ||
``` | ||
|
||
Remove any old `results.csv` and run benchmarks with `has-icu` flag enabled: | ||
|
||
``` | ||
$ rm results.csv | ||
$ cabal run bench --flag has-icu -- --csvraw=results.csv --quick | ||
``` | ||
|
||
The following command will now show the comparison between `text-icu` and | ||
`unicode-transforms`: | ||
|
||
``` | ||
$ cabal run chart --flag bench-show | ||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters