Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change Warmup in Harnesses to simply disregard measurements #249

Merged
merged 4 commits into from May 7, 2018

Conversation

1 participant
@smarr
Copy link
Owner

commented May 6, 2018

To avoid interference with benchmark execution, warmup should not be done separately from the main loop. Instead, it should be simply discarding measurements for the aggregate statistic.

This PR removes the extra warmup loops, and instead does not add the result of a run to the total as long as warmup is not completed.

@smarr smarr added the enhancement label May 6, 2018

@smarr smarr added this to the v0.7.0 milestone May 6, 2018

@smarr smarr added this to Open Issues in Completeness via automation May 6, 2018

smarr added some commits Apr 30, 2018

AsyncHarness: convert Warmup into simply ignoring results for aggrega…
…te statistics

Signed-off-by: Stefan Marr <git@stefan-marr.de>
Harness: convert Warmup into simply ignoring results for aggregate st…
…atistics

Signed-off-by: Stefan Marr <git@stefan-marr.de>

@smarr smarr force-pushed the warmup branch from ed9f63e to f5a3ccf May 6, 2018

smarr added some commits May 6, 2018

Update expected results
Signed-off-by: Stefan Marr <git@stefan-marr.de>
Adapt codespeed.conf for warmup semantics changes
Signed-off-by: Stefan Marr <git@stefan-marr.de>

@smarr smarr force-pushed the warmup branch from f5a3ccf to de8b4be May 7, 2018

@smarr smarr merged commit d3835ca into dev May 7, 2018

4 checks passed

Codacy/PR Quality Review Up to standards. A positive pull request.
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
continuous-integration/travis-ci/push The Travis CI build passed
Details
coverage/coveralls Coverage increased (+0.1%) to 77.544%
Details

Completeness automation moved this from Open Issues to Completed May 7, 2018

@smarr smarr deleted the warmup branch May 7, 2018

smarr referenced this pull request in richard-roberts/GraceLibrary May 7, 2018

New benchmarking harness with sum benchmark [note]
The harness can be used to execute a module for benchmarking. The harness first loads the module via the SOM platform and then extracts all methods, of the loaded module, whose signatures begin with "benchmark". Each method is then executed and the results of the benchmarked logged. The execution time can be present in either a human-readable form (use the argument `pretty`) or as CSV (use the argument `csv`).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.