Migrate the StreamProcessor tests to the new StreamPlatformExtension #10455
Assignees
Labels
kind/toil
Categorizes an issue or PR as general maintenance, i.e. cleanup, refactoring, etc.
version:8.1.0
Marks an issue as being completely or in parts released in 8.1.0
version:8.2.0-alpha1
Marks an issue as being completely or in parts released in 8.2.0-alpha1
version:8.2.0-alpha2
Marks an issue as being completely or in parts released in 8.2.0-alpha2
version:8.2.0
Marks an issue as being completely or in parts released in 8.2.0
Comments
Zelldon
added
the
kind/toil
This was referenced Sep 22, 2022
zeebe-bors-camunda bot
added a commit
that referenced
this issue
Sep 26, 2022
10443: Do not take a backup if it already exists r=deepthidevaki a=deepthidevaki ## Description After restore, the log is truncated to the checkpoint position. So the checkpoint record is processed again and will trigger a new backup with the same Id of the backup it restored from. With this PR, `BackupService` handles this case gracefully. In addition, we also do not take a new backup if existing backup is failed or in progress. Alternatively, we can delete this backup and take a new one. But chances of it happening (i.e triggering a new backup when one already is in progress/failed) is very low. So we can keep this simple. ## Related issues closes #10430 10450: fix(raft): handle exceptions on partition server init r=megglos a=megglos ## Description Previously any RuntimeException happening in RaftPartitionServer#initServer lead to a broken future chain during start which lead to a stale node without any logs on the actual exception occurred during init. Ultimately flying silently till [here](https://github.com/camunda/zeebe/blob/main/broker/src/main/java/io/camunda/zeebe/broker/bootstrap/PartitionManagerStep.java#L42) bringing the startup to a halt. With this change issues are transparent, see this [log](https://console.cloud.google.com/logs/query;cursorTimestamp=2022-09-22T11:20:44.904454673Z;query=resource.labels.namespace_name%3D%22medic-cw-37-de38e9e086-benchmark-mixed%22%0Aresource.labels.pod_name%3D%22medic-cw-37-de38e9e086-benchmark-mixed-zeebe-2%22%0A-resource.labels.container_name%3D%22debugger-9q4tw%22%0A-logName%3D%22projects%2Fzeebe-io%2Flogs%2Fevents%22%0Atimestamp%3D%222022-09-22T11:20:44.904454673Z%22%0AinsertId%3D%2238ntsbk0c2ikn344%22%0Atimestamp%3D%222022-09-22T11:20:44.904454673Z%22%0AinsertId%3D%2238ntsbk0c2ikn344%22;summaryFields=:false:32:beginning;timeRange=2022-09-22T10:20:44.905Z%2F2022-09-22T11:20:44.905Z?project=zeebe-io) from a pod created with this change. This bug was hiding the underlying issue a node not being able to start due to #10451 . ## Related issues relates to #10451 10458: Reorganize stream processor and engine tests r=Zelldon a=Zelldon ## Description Moved some tests around to make it easier to detect which need to be migrated for #10455 and to make it easier to create the new module and copy the tests, which are part of the StreamProcessor see #10130 <!-- Please explain the changes you made here. --> ## Related issues <!-- Which issues are closed by this PR or are related --> related to #10455 related to #10130 Co-authored-by: Deepthi Devaki Akkoorath <deepthidevaki@gmail.com> Co-authored-by: Meggle (Sebastian Bathke) <sebastian.bathke@camunda.com> Co-authored-by: Christopher Zell <zelldon91@googlemail.com>
zeebe-bors-camunda bot
added a commit
that referenced
this issue
Sep 26, 2022
10450: fix(raft): handle exceptions on partition server init r=megglos a=megglos ## Description Previously any RuntimeException happening in RaftPartitionServer#initServer lead to a broken future chain during start which lead to a stale node without any logs on the actual exception occurred during init. Ultimately flying silently till [here](https://github.com/camunda/zeebe/blob/main/broker/src/main/java/io/camunda/zeebe/broker/bootstrap/PartitionManagerStep.java#L42) bringing the startup to a halt. With this change issues are transparent, see this [log](https://console.cloud.google.com/logs/query;cursorTimestamp=2022-09-22T11:20:44.904454673Z;query=resource.labels.namespace_name%3D%22medic-cw-37-de38e9e086-benchmark-mixed%22%0Aresource.labels.pod_name%3D%22medic-cw-37-de38e9e086-benchmark-mixed-zeebe-2%22%0A-resource.labels.container_name%3D%22debugger-9q4tw%22%0A-logName%3D%22projects%2Fzeebe-io%2Flogs%2Fevents%22%0Atimestamp%3D%222022-09-22T11:20:44.904454673Z%22%0AinsertId%3D%2238ntsbk0c2ikn344%22%0Atimestamp%3D%222022-09-22T11:20:44.904454673Z%22%0AinsertId%3D%2238ntsbk0c2ikn344%22;summaryFields=:false:32:beginning;timeRange=2022-09-22T10:20:44.905Z%2F2022-09-22T11:20:44.905Z?project=zeebe-io) from a pod created with this change. This bug was hiding the underlying issue a node not being able to start due to #10451 . ## Related issues relates to #10451 10458: Reorganize stream processor and engine tests r=Zelldon a=Zelldon ## Description Moved some tests around to make it easier to detect which need to be migrated for #10455 and to make it easier to create the new module and copy the tests, which are part of the StreamProcessor see #10130 <!-- Please explain the changes you made here. --> ## Related issues <!-- Which issues are closed by this PR or are related --> related to #10455 related to #10130 Co-authored-by: Meggle (Sebastian Bathke) <sebastian.bathke@camunda.com> Co-authored-by: Christopher Zell <zelldon91@googlemail.com>
zeebe-bors-camunda bot
added a commit
that referenced
this issue
Sep 26, 2022
10458: Reorganize stream processor and engine tests r=Zelldon a=Zelldon ## Description Moved some tests around to make it easier to detect which need to be migrated for #10455 and to make it easier to create the new module and copy the tests, which are part of the StreamProcessor see #10130 <!-- Please explain the changes you made here. --> ## Related issues <!-- Which issues are closed by this PR or are related --> related to #10455 related to #10130 Co-authored-by: Christopher Zell <zelldon91@googlemail.com>
zeebe-bors-camunda bot
added a commit
that referenced
this issue
Sep 26, 2022
10458: Reorganize stream processor and engine tests r=Zelldon a=Zelldon ## Description Moved some tests around to make it easier to detect which need to be migrated for #10455 and to make it easier to create the new module and copy the tests, which are part of the StreamProcessor see #10130 <!-- Please explain the changes you made here. --> ## Related issues <!-- Which issues are closed by this PR or are related --> related to #10455 related to #10130 Co-authored-by: Christopher Zell <zelldon91@googlemail.com>
zeebe-bors-camunda bot
added a commit
that referenced
this issue
Sep 26, 2022
10458: Reorganize stream processor and engine tests r=Zelldon a=Zelldon ## Description Moved some tests around to make it easier to detect which need to be migrated for #10455 and to make it easier to create the new module and copy the tests, which are part of the StreamProcessor see #10130 <!-- Please explain the changes you made here. --> ## Related issues <!-- Which issues are closed by this PR or are related --> related to #10455 related to #10130 10482: deps(maven): bump snakeyaml from 1.32 to 1.33 r=Zelldon a=dependabot[bot] Bumps [snakeyaml](https://bitbucket.org/snakeyaml/snakeyaml) from 1.32 to 1.33. <details> <summary>Commits</summary> <ul> <li><a href="https://bitbucket.org/snakeyaml/snakeyaml/commits/eafb23ec31a0babe591c00e1b50e557a5e3f9a1d"><code>eafb23e</code></a> [maven-release-plugin] prepare for next development iteration</li> <li><a href="https://bitbucket.org/snakeyaml/snakeyaml/commits/26624702fab8e0a1c301d7fad723c048528f75c3"><code>2662470</code></a> Improve JavaDoc</li> <li><a href="https://bitbucket.org/snakeyaml/snakeyaml/commits/80827798f06aeb3d4f2632b94075ca7633418829"><code>8082779</code></a> Always emit numberish strings with quotes</li> <li><a href="https://bitbucket.org/snakeyaml/snakeyaml/commits/42d6c79430431fe9033d3ba50f6a7dc6798ba7ad"><code>42d6c79</code></a> Reformat test</li> <li><a href="https://bitbucket.org/snakeyaml/snakeyaml/commits/1962a437263348c3b90857cda4bbfa2bd97908f8"><code>1962a43</code></a> Refactor: rename variables in Emitter</li> <li><a href="https://bitbucket.org/snakeyaml/snakeyaml/commits/bc594ad6e2b87c3fc26844e407276796fd866a40"><code>bc594ad</code></a> Issue 553: honor code point limit in loadAll</li> <li><a href="https://bitbucket.org/snakeyaml/snakeyaml/commits/c3e98fd755a949f65cf11f2ff39e55a1c2afd1c2"><code>c3e98fd</code></a> Update changes.xml</li> <li><a href="https://bitbucket.org/snakeyaml/snakeyaml/commits/a06f76859f2f07580b1d9fa6b66ea84aaad26cf8"><code>a06f768</code></a> Remove deprecated Tag manipulation</li> <li><a href="https://bitbucket.org/snakeyaml/snakeyaml/commits/5a0027a3781b92f59bf92cdeb1b7590589993efd"><code>5a0027a</code></a> Remove unused WhitespaceToken</li> <li><a href="https://bitbucket.org/snakeyaml/snakeyaml/commits/3f05838828b8df36ab961bf836f373b8c20cb8ff"><code>3f05838</code></a> Improve JavaDoc</li> <li>Additional commits viewable in <a href="https://bitbucket.org/snakeyaml/snakeyaml/branches/compare/snakeyaml-1.33..snakeyaml-1.32">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting ``@dependabot` rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - ``@dependabot` rebase` will rebase this PR - ``@dependabot` recreate` will recreate this PR, overwriting any edits that have been made to it - ``@dependabot` merge` will merge this PR after your CI passes on it - ``@dependabot` squash and merge` will squash and merge this PR after your CI passes on it - ``@dependabot` cancel merge` will cancel a previously requested merge and block automerging - ``@dependabot` reopen` will reopen this PR if it is closed - ``@dependabot` close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - ``@dependabot` ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - ``@dependabot` ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - ``@dependabot` ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Co-authored-by: Christopher Zell <zelldon91@googlemail.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
zeebe-bors-camunda bot
added a commit
that referenced
this issue
Sep 26, 2022
10458: Reorganize stream processor and engine tests r=Zelldon a=Zelldon ## Description Moved some tests around to make it easier to detect which need to be migrated for #10455 and to make it easier to create the new module and copy the tests, which are part of the StreamProcessor see #10130 <!-- Please explain the changes you made here. --> ## Related issues <!-- Which issues are closed by this PR or are related --> related to #10455 related to #10130 Co-authored-by: Christopher Zell <zelldon91@googlemail.com>
zeebe-bors-camunda bot
added a commit
that referenced
this issue
Sep 26, 2022
10458: Reorganize stream processor and engine tests r=Zelldon a=Zelldon ## Description Moved some tests around to make it easier to detect which need to be migrated for #10455 and to make it easier to create the new module and copy the tests, which are part of the StreamProcessor see #10130 <!-- Please explain the changes you made here. --> ## Related issues <!-- Which issues are closed by this PR or are related --> related to #10455 related to #10130 Co-authored-by: Christopher Zell <zelldon91@googlemail.com>
zeebe-bors-camunda bot
added a commit
that referenced
this issue
Sep 29, 2022
10458: Reorganize stream processor and engine tests r=Zelldon a=Zelldon ## Description Moved some tests around to make it easier to detect which need to be migrated for #10455 and to make it easier to create the new module and copy the tests, which are part of the StreamProcessor see #10130 <!-- Please explain the changes you made here. --> ## Related issues <!-- Which issues are closed by this PR or are related --> related to #10455 related to #10130 Co-authored-by: Christopher Zell <zelldon91@googlemail.com>
zeebe-bors-camunda bot
added a commit
that referenced
this issue
Sep 29, 2022
10458: Reorganize stream processor and engine tests r=Zelldon a=Zelldon ## Description Moved some tests around to make it easier to detect which need to be migrated for #10455 and to make it easier to create the new module and copy the tests, which are part of the StreamProcessor see #10130 <!-- Please explain the changes you made here. --> ## Related issues <!-- Which issues are closed by this PR or are related --> related to #10455 related to #10130 10556: fix(helm): rename podSecurityContext to containerSecurityContext r=Zelldon a=megglos ## Description The recent v8.0.14 release introduced containerSecurityContext, see camunda/camunda-platform-helm#374 and was actually failing now with the format `podSecurityContext` was set. ``` Error: INSTALLATION FAILED: unable to build kubernetes objects from release manifest: error validating "": error validating data: ValidationError(StatefulSet.spec.template.spec.securityContext): unknown field "capabilities" in io.k8s.api.core.v1.PodSecurityContext ``` //cc `@aabouzaid` Co-authored-by: Christopher Zell <zelldon91@googlemail.com> Co-authored-by: Meggle (Sebastian Bathke) <sebastian.bathke@camunda.com>
zeebe-bors-camunda bot
added a commit
that referenced
this issue
Sep 29, 2022
10458: Reorganize stream processor and engine tests r=Zelldon a=Zelldon ## Description Moved some tests around to make it easier to detect which need to be migrated for #10455 and to make it easier to create the new module and copy the tests, which are part of the StreamProcessor see #10130 <!-- Please explain the changes you made here. --> ## Related issues <!-- Which issues are closed by this PR or are related --> related to #10455 related to #10130 Co-authored-by: Christopher Zell <zelldon91@googlemail.com>
zeebe-bors-camunda bot
added a commit
that referenced
this issue
Sep 29, 2022
10468: Migrate StreamProcessor test r=Zelldon a=Zelldon
## Description
Migrate the StreamProcessor to the new StreamProcessorExtension and Junit5.
I completely iterated over all tests and rewrote them.
This is part of #10455 which is necessary to split up into StreamPlatform into a separate module (#10130 )
Blocked by #10458
<!-- Please explain the changes you made here. -->
## Related issues
<!-- Which issues are closed by this PR or are related -->
related to #10455
Co-authored-by: Christopher Zell <zelldon91@googlemail.com>
Zelldon
added
the
version:8.1.0
10 tasks
korthout
added
the
version:8.2.0-alpha1
This was referenced Nov 1, 2022
Merged
zeebe-bors-camunda bot
added a commit
that referenced
this issue
Nov 2, 2022
10876: Migrate replay test r=Zelldon a=Zelldon ## Description Migrate the StreamProcessorReplayTest.java to JUNIT 5 and makes use of the Streamplatform extension. Rewrite all test cases and add two new ones. Both new tests verify whether the restore work as expected. One test from log and the other from snapshot. | OLD | NEW | |--------|---------| ||| Refactoring the test also reduced the execution time by ~1/4. <!-- Please explain the changes you made here. --> ## Related issues <!-- Which issues are closed by this PR or are related --> related to #10455 Co-authored-by: Christopher Zell <zelldon91@googlemail.com>
|](https://user-images.githubusercontent.com/2758593/199228219-e3be4914-e8aa-40b9-ba14-e1839f65c8a1.png)||){kind=link}
zeebe-bors-camunda bot
added a commit
that referenced
this issue
Nov 2, 2022
10876: Migrate replay test r=Zelldon a=Zelldon ## Description Migrate the StreamProcessorReplayTest.java to JUNIT 5 and makes use of the Streamplatform extension. Rewrite all test cases and add two new ones. Both new tests verify whether the restore work as expected. One test from log and the other from snapshot. | OLD | NEW | |--------|---------| ||| Refactoring the test also reduced the execution time by ~1/4. <!-- Please explain the changes you made here. --> ## Related issues <!-- Which issues are closed by this PR or are related --> related to #10455 Co-authored-by: Christopher Zell <zelldon91@googlemail.com>
This was referenced Nov 2, 2022
zeebe-bors-camunda bot
added a commit
that referenced
this issue
Nov 3, 2022
10888: Migrate the TypedStreamProcessorTest r=Zelldon a=Zelldon ## Description * Rewrote [shouldWriteSourceEventAndProducerOnBatch](dbc19f6) test and move it to StreamProcessorTest * Extend StreamProcessorTest: * Add test for writing rejection on error handling * Split a test such that we test separately whether processing can continue and we can write a response on error * [iterate over TypedStreamProcessorTest](f6aa792) * The test makes still sense from the engine perspective since we can verify whether commands which fail to process are automatically rejected etc. * <!-- Please explain the changes you made here. --> ## Related issues <!-- Which issues are closed by this PR or are related --> related to #10455 Co-authored-by: Christopher Zell <zelldon91@googlemail.com>
zeebe-bors-camunda bot
added a commit
that referenced
this issue
Nov 3, 2022
10888: Migrate the TypedStreamProcessorTest r=Zelldon a=Zelldon ## Description * Rewrote [shouldWriteSourceEventAndProducerOnBatch](dbc19f6) test and move it to StreamProcessorTest * Extend StreamProcessorTest: * Add test for writing rejection on error handling * Split a test such that we test separately whether processing can continue and we can write a response on error * [iterate over TypedStreamProcessorTest](f6aa792) * The test makes still sense from the engine perspective since we can verify whether commands which fail to process are automatically rejected etc. * <!-- Please explain the changes you made here. --> ## Related issues <!-- Which issues are closed by this PR or are related --> related to #10455 Co-authored-by: Christopher Zell <zelldon91@googlemail.com>
zeebe-bors-camunda bot
added a commit
that referenced
this issue
Nov 3, 2022
10892: Merge SkipFailingEventsTest with EngineErrorHandlingTest r=Zelldon a=Zelldon ## Description ~:x: Blocked by #10888 Merge SkipFailingEventsTest with EngineErrorHandlingTest, to make more clear what they actually testing and in which responsibility this test lies. <!-- Please explain the changes you made here. --> ## Related issues <!-- Which issues are closed by this PR or are related --> related #10455 Co-authored-by: Christopher Zell <zelldon91@googlemail.com>
zeebe-bors-camunda bot
added a commit
that referenced
this issue
Nov 3, 2022
10879: Migrate the replay mode test r=Zelldon a=Zelldon ## Description ~:x: blocked by #10876~ Migrates the `StreamProcessorReplayModeTest` to JUNIT 5 and uses StreamPlatform extension. The new test is called `StreamProcessorContinouslyReplayModeTest.java` Similar to #10876 the test execution has been improved. Some additional improvements have been made to verify whether the state is restored and contains the last processed position etc. | OLD | NEW | |--------|--------| ||| One test has been removed since it is already tested in another class and one has been removed since the test doesn't make sense like that. <!-- Please explain the changes you made here. --> ## Related issues <!-- Which issues are closed by this PR or are related --> related #10455 Co-authored-by: Christopher Zell <zelldon91@googlemail.com>
|](https://user-images.githubusercontent.com/2758593/199256394-0733eba3-bfe3-4b57-8ad8-02300a07b400.png)||){kind=link}
zeebe-bors-camunda bot
added a commit
that referenced
this issue
Nov 3, 2022
10892: Merge SkipFailingEventsTest with EngineErrorHandlingTest r=Zelldon a=Zelldon ## Description ~:x: Blocked by #10888 Merge SkipFailingEventsTest with EngineErrorHandlingTest, to make more clear what they actually testing and in which responsibility this test lies. <!-- Please explain the changes you made here. --> ## Related issues <!-- Which issues are closed by this PR or are related --> related #10455 Co-authored-by: Christopher Zell <zelldon91@googlemail.com>
zeebe-bors-camunda bot
added a commit
that referenced
this issue
Nov 4, 2022
10894: Remove StreamProcessorReprocessingTest r=Zelldon a=Zelldon ## Description :x: blocked by #10879 <!-- Please explain the changes you made here. --> Deletes test and verifies for each test method, whether we already have an replacement. ## Related issues <!-- Which issues are closed by this PR or are related --> related to #10455 Co-authored-by: Christopher Zell <zelldon91@googlemail.com>
remcowesterhoud
added
the
version:8.2.0-alpha2
npepinpe
added
the
version:8.2.0
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Description
We recently introduced a new StreamPlatform extension (#10036) which allows us to write tests for the StreamProcessor without the need of setting up the engine etc.
We need to migrate our tests in order to be able to reduce the dependency on the EventAppliers related (79bf2bc), which has been reverted with #10028 because right now all our tests depends on it see #10027
This is blocking the creation of the StreamProcessor Module #10130
blocking #10130
ContinuouslyReplayTestMore related to engine stateReplayStateRandomizedPropertyTestReplayStateTestThe text was updated successfully, but these errors were encountered: