Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GEODE-6086: Adding a tool to allow command line analysis of test runs #7

Merged
merged 3 commits into from Nov 28, 2018

Conversation

WireBaron
Copy link

This change adds the analyzeRun gradle target, which, when passed a baseline
result directory and test result directory via --args, will analyze the output
and report the variation of the test from the baseline.

Sample output:

MacBook-Pro-2:geode-performance browe$ ./gradlew analyzeRun --args "/Users/browe/project/geode-performance/geode-benchmarks/output2 /Users/browe/project/geode-performance/geode-benchmarks/output"

> Task :harness:analyzeRun
Running analyzer
Comparing test result at /Users/browe/project/geode-performance/geode-benchmarks/output2 to baseline at /Users/browe/project/geode-performance/geode-benchmarks/output
-- PartitionedPutBenchmark --
average ops/second
Result: 47386.93220338983
Baseline: 46237.745762711864
Relative performance: 1.0248538595842343

99th percentile latency
Result: 292.85714285714283
Baseline: 294.11764705882354
Relative performance: 0.9957142857142857

Two remaining pain points I'd like to address:

  1. When adding a benchmark it would be easier to use roles rather than node names
  2. It would also be nice to reference the benchmark classes directly in the analyzer (to get their output directories), but that dependency goes the wrong way.

This change adds the analyzeRun gradle target, which, when passed a baseline
result directory and test result directory via --args, will analyze the output
and report the variation of the test from the baseline.
@WireBaron WireBaron merged commit 30ae13c into apache:develop Nov 28, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
1 participant