Add some version-store microbenchmarks #6952
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
snazy
force-pushed
the
version-store-microbench
branch
2 times, most recently
from
15d4151
to
5fdd4b7
Compare
This can serve as a tool to later optimize these parts in the Nessie code base: * version store implementation * database specific `Persist` implementations * storate logic implementations So far, the results look good enough, no concerning outliers w/ in-memory, mongo + cassandra. Microbenchmarks use `BackendTestFactory` to locate and initialize a `Backend` (and in turn `Persist`) implementation. Microbenchmarks are intended to inspect the Nessie code down up to the database level, but not intended to optimize configurations for a particular database. In other words: only the Nessie code is interesting, but the performance of the database itself is not (that much). The `BackendTestFactory` interface got a new function `getName()` and the corresponding service definitions have been added. Needed to use a new Gradle project, because the microbenchmarks need the Java `ServiceLoader` for various reasons, so it the JMH uber-jar needs the _shadow_ plugin, but its presence would make `nessie-services.jar` an uber-jar as well - so it's easier to use a separate Gradle project.
snazy
force-pushed
the
version-store-microbench
branch
from
5fdd4b7
to
4c649ce
Compare
adutra
previously approved these changes
Jun 5, 2023
servers/services-bench/README.md
Outdated
|
|
||
| ## Linux Perf tools | ||
|
|
||
| Install the appropriate `linux-tools` package for your distribution, or `make` it from the Linux sources in `tools/perf` |
There was a problem hiding this comment.
From what I could find, the linux-tools package is a Ubuntu thing. Maybe we could rephrase this paragraph and focus more on which commands need to be available, rather than which packages contain those commands.
servers/services-bench/README.md
Outdated
|
|
||
| Running (Linux): | ||
| ```bash | ||
| LD_LIBRARY_PATH=(PATH-TO-LIBRARY)/ java \ |
There was a problem hiding this comment.
FYI, the only command that worked for me on macOS was:
java -Djava.library.path=/path/to/async-profiler-2.9-macos/build/ -jar servers/services-bench/build/libs/nessie-services-bench-*-jmh.jar -prof asyncNeither LD_LIBRARY_PATH nor DYLD_LIBRARY_PATH worked.
adutra
approved these changes
Jun 5, 2023
snazy
added a commit
to snazy/nessie
that referenced
this pull request
Jun 5, 2023
snazy
added a commit
to snazy/nessie
that referenced
this pull request
Jun 5, 2023
snazy
added a commit
that referenced
this pull request
Jun 5, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This can serve as a tool to later optimize these parts in the Nessie code base:
PersistimplementationsSo far, the results look good enough, no concerning outliers w/ in-memory, mongo + cassandra.
Microbenchmarks use
BackendTestFactoryto locate and initialize aBackend(and in turnPersist) implementation. Microbenchmarks are intended to inspect the Nessie code down up to the database level, but not intended to optimize configurations for a particular database. In other words: only the Nessie code is interesting, but the performance of the database itself is not (that much).The
BackendTestFactoryinterface got a new functiongetName()and the corresponding service definitions have been added.