Skip to content

Commit

Permalink
Toy external R benchmark
Browse files Browse the repository at this point in the history
  • Loading branch information
dianaclarke committed Jun 23, 2021
1 parent 6bea338 commit d58b273
Show file tree
Hide file tree
Showing 2 changed files with 21 additions and 6 deletions.
21 changes: 16 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -661,7 +661,16 @@ Benchmark result:

### Example R benchmarks

A few examples ilustrating how to integrate R benchmarks with Conbench.
A few examples illustrating how to integrate R benchmarks with Conbench.

The first one just times `1 + 1` in R, and the second one executes an R
benchmark from a library of R benchmarks (in this case
[arrowbench](https://github.com/ursacomputing/arrowbench)).

If you find yourself wrapping a lot of R benchmarks in Python to integrate them
with Conbench (to get uniform JSON benchmark results which you can persist and
publish on a Conbench serve), you'll probably want to extract much of the
boilerplate out into a base class.


```python
Expand Down Expand Up @@ -703,7 +712,7 @@ class ExternalBenchmarkR(conbench.runner.Benchmark):


```
cd ~/workspace/conbench/conbench/tests/benchmark/
$ cd ~/workspace/conbench/conbench/tests/benchmark/
$ conbench external-r --help
Usage: conbench external-r [OPTIONS]
Expand Down Expand Up @@ -769,14 +778,16 @@ class ExternalBenchmarkOptionsR(conbench.runner.Benchmark):
```

```
cd ~/workspace/conbench/conbench/tests/benchmark/
$ cd ~/workspace/conbench/conbench/tests/benchmark/
$ conbench external-r --help
Usage: conbench external-r [OPTIONS]
Usage: conbench external-r-options [OPTIONS]
Run external-r benchmark.
Run external-r-options benchmark.
Options:
--iterations INTEGER [default: 1]
--drop-caches BOOLEAN [default: False]
--show-result BOOLEAN [default: True]
--show-output BOOLEAN [default: False]
--run-id TEXT Group executions together with a run id.
Expand Down
6 changes: 5 additions & 1 deletion conbench/tests/benchmark/test_cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -280,7 +280,11 @@ def test_conbench_command_external_options_r(runner):
command = "external-r-options --show-result=false --show-output=true"
with unittest.mock.patch("conbench.util.Connection.publish"):
result = runner.invoke(conbench, command)
assert_command_contains(result, '"result"')

try:
assert_command_contains(result, '"result"')
except:
pytest.skip("Probably no arrowbench")


def test_conbench_command_external_options_r_help(runner):
Expand Down

0 comments on commit d58b273

Please sign in to comment.