Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add infrastructure support for microbenchmarking #18242

Closed
danielmitterdorfer opened this issue May 10, 2016 · 0 comments

Comments

Projects
None yet
2 participants
@danielmitterdorfer
Copy link
Member

commented May 10, 2016

Rationale

We have recently removed a few benchmarks from Elasticsearch core (see #15356). Although most of our benchmarking should be on the macro-level (i.e. running against a real Elasticsearch cluster) we're sometimes interested in benchmarking at a more narrow level. As we don't need a fully running cluster in that case, it makes sense to base them on JMH as it is the de-facto standard for microbenchmarking on the JVM.

Scope

The scope of this ticket is just to add the necessary infrastructure to our build scripts (and maybe provide one or two example benchmarks). We will not recreate the former benchmark suite in this ticket (nor should we aim to recreate the whole suite again).

Next Steps

After this is set up, we should run the benchmarks also regularly on Jenkins (on a dedicated quiet machine for benchmarks) and add proper reporting, e.g. with the JMH Benchmark Jenkins Plugin to spot performance regressions.

@danielmitterdorfer danielmitterdorfer self-assigned this May 10, 2016

danielmitterdorfer added a commit to danielmitterdorfer/elasticsearch that referenced this issue Jun 15, 2016

Add microbenchmarking infrastructure
With this commit we add a benchmarks project that contains the necessary build
infrastructure and an example benchmark. It is added as a separate project to avoid
interfering with the regular build too much (especially sanity checks) and to keep
the microbenchmarks isolated.

Microbenchmarks are generated with `gradle :benchmarks:jmhJar` and can be run with
` gradle :benchmarks:jmh`.

We intentionally do not use the
[jmh-gradle-plugin](https://github.com/melix/jmh-gradle-plugin) as it causes all
sorts of problems (dependencies are not properly excluded, not all JMH parameters
can be set) and it adds another abstraction layer that is not needed.

Closes elastic#18242

danielmitterdorfer added a commit that referenced this issue Jun 15, 2016

Add microbenchmarking infrastructure (#18891)
With this commit we add a benchmarks project that contains the necessary build
infrastructure and an example benchmark. It is added as a separate project to avoid
interfering with the regular build too much (especially sanity checks) and to keep
the microbenchmarks isolated.

Microbenchmarks are generated with `gradle :benchmarks:jmhJar` and can be run with
` gradle :benchmarks:jmh`.

We intentionally do not use the
[jmh-gradle-plugin](https://github.com/melix/jmh-gradle-plugin) as it causes all
sorts of problems (dependencies are not properly excluded, not all JMH parameters
can be set) and it adds another abstraction layer that is not needed.

Closes #18242

@danielmitterdorfer danielmitterdorfer removed their assignment Jun 16, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.