Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

A broker can take backup #10262

Closed
Tracked by #9606
deepthidevaki opened this issue Sep 2, 2022 · 0 comments · Fixed by #10374
Closed
Tracked by #9606

A broker can take backup #10262

deepthidevaki opened this issue Sep 2, 2022 · 0 comments · Fixed by #10374
Assignees
@deepthidevaki
Labels
kind/toil Categorizes an issue or PR as general maintenance, i.e. cleanup, refactoring, etc. version:8.1.0 Marks an issue as being completely or in parts released in 8.1.0

Comments

@deepthidevaki
Copy link
Contributor

Description

This is the issue for putting it all together. The goal of this issue is to connect all necessary components, tests if everything works and find missing pieces and bugs. Once this issue is done, a broker should take backup on all partitions.

  • The broker should take backup on receiving a backup request via BackupApiRequestHandler
  • Inter partition communication should trigger backups on other partitions

Note:- This issue does not cover the gateway. Without the gateway we might not be able to write a full integration test. But to find bugs as early as possible, we should already do minimum tests.
@deepthidevaki deepthidevaki added the kind/toil Categorizes an issue or PR as general maintenance, i.e. cleanup, refactoring, etc. label Sep 2, 2022
This was referenced Sep 2, 2022
Closed
Merged
Merged
@deepthidevaki deepthidevaki self-assigned this Sep 2, 2022
zeebe-bors-camunda bot added a commit that referenced this issue Sep 5, 2022
@zeebe-bors-camunda @deepthidevaki
merge: #10270 #10271
fca38d2 
10270: fix(streamprocessor): stream processor can read checkpoint record values r=deepthidevaki a=deepthidevaki

## Description

StreamProcessor uses this event registry to read the value based on the given ValueType.

This event registry and `RecordValues` that reads the record is now part of engine. It is not ideal to add checkpoint value types in the engine, as the engine shouldn't care about it. Alternatively, I tried to move `RecordValues` to StreamProcessor. The goal was that each `RecordProcessor` can define its own registry which we can feed it into `RecordValues`. However, this did not work because engine uses `RecordValues` in their tests. This will create a dependency from engine to streamprocessor which is not allowed. Engine should only depend on the common api. Besides, ExporterDirector also used `RecordValues` to read the record.

Because of these dependencies, changing anything here would required a lot of refactoring. So as a quick fix, I have added the checkpoint value type to the registry. It does not add any additional dependencies to the engine.

If this is ok, I will create a follow up issue to refactor this in engine-abstraction topic.

## Related issues

related #10262 



10271: fix(broker): notify partition command sender and reciever on new checkpoint r=deepthidevaki a=deepthidevaki

## Description

`InterPartitionCommandSender` and Receiver should keep track of the latest checkpoint. This is done via `CheckpointListener`. This PR register them as listeners immediately after they are installed during partition transition.

## Related issues

related #10262 



Co-authored-by: Deepthi Devaki Akkoorath <deepthidevaki@gmail.com>
@deepthidevaki deepthidevaki mentioned this issue Sep 15, 2022
Merged
10 tasks
zeebe-bors-camunda bot added a commit that referenced this issue Sep 16, 2022
@zeebe-bors-camunda @dependabot @deepthidevaki
merge: #10332 #10374
57da965 
10332: deps(maven): bump rest-assured from 5.1.1 to 5.2.0 r=npepinpe a=dependabot[bot]

Bumps [rest-assured](https://github.com/rest-assured/rest-assured) from 5.1.1 to 5.2.0.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a href="https://github.com/rest-assured/rest-assured/blob/master/changelog.txt">rest-assured's changelog</a>.</em></p>
<blockquote>
<h2>Changelog 5.2.0 (2022-09-09)</h2>
<ul>
<li>
<p>Improved FilterContext used in Filters by adding the method FilterContext#hasValue(name, object). This makes it easier to check if a value exists <em>and</em> is equal to the expect object.</p>
</li>
<li>
<p>Introducing a much improved CSRF (cross-site request forgery) support. For example:
given().
csrf(&quot;/users&quot;).
formParm(&quot;firstName&quot;, &quot;John&quot;).
formParm(&quot;lastName&quot;, &quot;Doe&quot;).
when().
post(&quot;/users&quot;).
then().
statusCode(200);</p>
<p>This will first make a GET request to /users (due to csrf(&quot;/users&quot;)) to get an HTML page that contains the CSRF token.
Rest Assured will then automatically try to find the input field that contains the CSRF token and include in the POST to /users.</p>
<p>Here's an example of what Rest Assured expects as a response for the GET request to /users:</p>
<!-- raw HTML omitted -->
</li>
<li>
<p>Fixed so that form authentication takes CSRF into account. The previous form authentication CSRF implementation didn't really work (sorry!).
Now you can combine csrf with form authentication and it actually works as expected! Note that for requests other than GET or HEAD,
you need to specify <em>both</em> form authentication <em>and</em> csrf, e.g.</p>
<p>given().
csrf(&quot;/users&quot;).
formParm(&quot;firstName&quot;, &quot;John&quot;).
formParm(&quot;lastName&quot;, &quot;Doe&quot;).</p>
</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="https://github.com/rest-assured/rest-assured/commit/c8e4ca53e4e0bc3ee761b5dbb07100b5c9fd74ae"><code>c8e4ca5</code></a> [maven-release-plugin] prepare release rest-assured-5.2.0</li>
<li><a href="https://github.com/rest-assured/rest-assured/commit/d72c03e0478d9b5de64aedba8ca87a5ad08c1e35"><code>d72c03e</code></a> [ci skip] Updated changelog with release date</li>
<li><a href="https://github.com/rest-assured/rest-assured/commit/0db04cb508568cede7914cb88e09cf8d25f6bddb"><code>0db04cb</code></a> Fixed imports</li>
<li><a href="https://github.com/rest-assured/rest-assured/commit/0cda32928e18acc88aceeac902227048f3378881"><code>0cda329</code></a> Fixed failing test</li>
<li><a href="https://github.com/rest-assured/rest-assured/commit/34c857a22276b823bb2805a1f04aad38d638c6cc"><code>34c857a</code></a> Moving sundr-maven-plugin to release profile since it doesn't seem to work on...</li>
<li><a href="https://github.com/rest-assured/rest-assured/commit/42d45528ddfad4e77a7aec4028fc6ce9ccdf388b"><code>42d4552</code></a> Cleanup</li>
<li><a href="https://github.com/rest-assured/rest-assured/commit/4c207e887971cbd894884374aca3c361bc43edb7"><code>4c207e8</code></a> Upgrading sundr-maven-plugin to 0.93.0</li>
<li><a href="https://github.com/rest-assured/rest-assured/commit/246ba8740607e23f00097e04526a019685f22627"><code>246ba87</code></a> Added csrf to RequestSpecBuilder.java</li>
<li><a href="https://github.com/rest-assured/rest-assured/commit/a6216b75f9f05e1cb2e5d4b747999af3bb663ac8"><code>a6216b7</code></a> Fixed some bugs and made some improvements to CSRF</li>
<li><a href="https://github.com/rest-assured/rest-assured/commit/50ad97a8f415c02f1f818521d243c0097a5075a2"><code>50ad97a</code></a> Upgraded kotlin module to using Kotlin 1.7.10 (previously 1.6.21 was used)</li>
<li>Additional commits viewable in <a href="https://github.com/rest-assured/rest-assured/compare/rest-assured-5.1.1...rest-assured-5.2.0">compare view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=io.rest-assured:rest-assured&package-manager=maven&previous-version=5.1.1&new-version=5.2.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting ``@dependabot` rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- ``@dependabot` rebase` will rebase this PR
- ``@dependabot` recreate` will recreate this PR, overwriting any edits that have been made to it
- ``@dependabot` merge` will merge this PR after your CI passes on it
- ``@dependabot` squash and merge` will squash and merge this PR after your CI passes on it
- ``@dependabot` cancel merge` will cancel a previously requested merge and block automerging
- ``@dependabot` reopen` will reopen this PR if it is closed
- ``@dependabot` close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- ``@dependabot` ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- ``@dependabot` ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- ``@dependabot` ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


</details>

10374: Verify broker can take backup r=deepthidevaki a=deepthidevaki

## Description

- Add a simple test to verify broker can take backup.
- Fixed several minor bugs that were found with this test. 

This PR only adds a basic test to verify backup. More scenarios should be tested later.

Also added a Junit5 extension to replace `ClusteringRule`. Now it is a wrapper around `ClusteringRule`. Once all tests have been migrated to junit5, we would be able to replace it.

## Related issues

closes #10262 



Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Deepthi Devaki Akkoorath <deepthidevaki@gmail.com>
@Zelldon Zelldon added the version:8.1.0 Marks an issue as being completely or in parts released in 8.1.0 label Oct 4, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@deepthidevaki deepthidevaki

Labels
kind/toil Categorizes an issue or PR as general maintenance, i.e. cleanup, refactoring, etc. version:8.1.0 Marks an issue as being completely or in parts released in 8.1.0
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Verify broker can take backup camunda/camunda
2 participants
@deepthidevaki @Zelldon