-
Notifications
You must be signed in to change notification settings - Fork 557
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Performance test for large state and stable performance #13121
Conversation
270a515
to
0ffbbe4
Compare
In order to be able to easily exchange or replace the StreamProcessorRule a new interface is introduced, which abstracts the writing of commands away. The implementation can be completelty separated and change without breaking the test clients.
Extract class ProcessingExporterTransistor from EngineRule in order to be used by Performance Test
Create TestInterPartitionCommandSender.java in util package, in order to be used by both EngineRule and performance tests.
0ffbbe4
to
50f23ca
Compare
Example execution with
|
Example with SST partitioning disabled
|
* use separated ProcessingExporterTransistor * use separated TestInterPartitionCommandSender.java * remove unused ReprocessingCompletedListener
Create new JMH test in order to verify stable performance on large state. * Create helper class TestEngine.java in order to simplify and encapsulate the engine setup. * Create record TestContext.java which contains infrastructure related dependencies, which might be shared by multiple engines. * Add JMH benchmark which allows to measure how the throughput looks like of process instance creation. * Use Seconds as output unit, for better representation and understandability * Use measure timeunit seconds this means the creation of PI is tried as often as possible in one second * Increase warm up and measure iterations * Reset the RecordingExporter in between, in order to avoid slowing down the benchmark * State is created on setup, to make sure we always create the same amount of data
50f23ca
to
a53aa52
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great job @Zelldon 🚀
Most of my comments are a bit nit-picky and actually unrelated to the JMH benchmark.
❓ I was only able to run these once through IntelliJ, every run after the first is seemingly stuck in the warmup phase. I guess that's not the case for you?
engine/src/test/java/io/camunda/zeebe/engine/util/ProcessingExporterTransistor.java
Outdated
Show resolved
Hide resolved
engine/src/test/java/io/camunda/zeebe/engine/util/ProcessingExporterTransistor.java
Show resolved
Hide resolved
engine/src/test/java/io/camunda/zeebe/engine/util/EngineRule.java
Outdated
Show resolved
Hide resolved
engine/src/test/java/io/camunda/zeebe/engine/util/EngineRule.java
Outdated
Show resolved
Hide resolved
engine/src/test/java/io/camunda/zeebe/engine/perf/TestContext.java
Outdated
Show resolved
Hide resolved
bors r+ |
/** | ||
* Containing infrastructure related dependencies which might be shared between TestEngines. | ||
* | ||
* @param actorScheduler the scheduler which is used during tests |
Check notice
Code scanning / CodeQL
Spurious Javadoc @param tags Note
* Containing infrastructure related dependencies which might be shared between TestEngines. | ||
* | ||
* @param actorScheduler the scheduler which is used during tests | ||
* @param temporaryFolder the temporary folder where the log and runtime is written to |
Check notice
Code scanning / CodeQL
Spurious Javadoc @param tags Note
* | ||
* @param actorScheduler the scheduler which is used during tests | ||
* @param temporaryFolder the temporary folder where the log and runtime is written to | ||
* @param autoCloseableRule a collector of all to managed resources, which should be cleaned up |
Check notice
Code scanning / CodeQL
Spurious Javadoc @param tags Note
Build succeeded: |
Description
@oleschoenburg I created this PR in order to get the first increment merged and to not overwhelm you with more changes. Most of the changes here are refactorings and preparation for the JMH test, plus of course the JMH test itself.
I applied already several things we have discussed, like creation of large state in setup phase, keeping the test in engine module etc. In an upcoming PR I will execute the JMH benchmark inside a unit test, which is executed in a separate CI job, please see related comment.
Related issues
related to #12241