KAFKA-14650: Synchronize access to tasks inside task manager #13167
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
ping @cadonna for review. |
lucasbru
reviewed
Jan 30, 2023
Make LeaderState's grantingVoters field explicitly immutable. The set of voters that granted their voter to the current leader was already immutable. This change makes that explicit. Reviewers: Jason Gustafson <jason@confluent.io>, Mathew Hogan <mathewdhogan@@users.noreply.github.com>
…apache#13131) Reviewers: Mickael Maison <mickael.maison@gmail.com>, Christo Lolov <christololov@gmail.com>, Sagar Rao <sagarmeansocean@gmail.com>
…config definitions (apache#13148) Reviewers: Mickael Maison <mickael.maison@gmail.com>, Greg Harris <gharris1727@gmail.com>
When running junit tests, it is not good to block forever on CompletableFuture objects. When there are bugs, this can lead to junit tests hanging forever. Jenkins does not deal with this well -- it often brings down the whole multi-hour test run. Therefore, when running integration tests in JUnit, set some reasonable time limits on broker and controller startup time. Reviewers: Jason Gustafson <jason@confluent.io>
When in migration-from-ZK mode and sending RPCs to ZK-based brokers, the KRaft controller must send full UpdateMetadataRequests prior to sending full LeaderAndIsrRequests. If the controller sends the requests in the other order, and the ZK-based broker does not already know about some of the nodes referenced in the LeaderAndIsrRequest, it will reject the request. This PR includes an integration test, and a number of other small fixes for dual-write. Co-authored-by: Akhilesh C <akhileshchg@users.noreply.github.com> Reviewers: Colin P. McCabe <cmccabe@apache.org>
This PR adds a safe-guard for divide-by-zero. While `totalCapacity` can never be zero, an explicit error message is desirable. Reviewers: Bill Bejeck <bill@confluent.io>, Guozhang Wang <guozhang@confluent.io>
…pache#13142) This PR refactors how the list of open iterators for RocksDB stores is managed. Prior to this PR, the `openIterators` list was passed into the constructor for the iterators themselves, allowing `RocksDbIterator.close()` to remove the iterator from the `openIterators` list. After this PR, the iterators themselves will not know about lists of open iterators. Instead, a generic close callback is exposed, and it's the responsibility of the store that creates a new iterator to set the callback on the iterator, to ensure that closing an iterator removes the iterator from the list of open iterators. This refactor is desirable because it enables more flexible iterator lifecycle management. Building on top of this, RocksDBStore is updated with an option to allow the user (i.e., the caller of methods such as `range()` and `prefixScan()` which return iterators) to pass a custom `openIterators` list for the new iterator to be stored in. This will allow for a new Segments implementation where multiple Segments can share the same RocksDBStore instance, while having each Segment manage its own open iterators. Part of KIP-889. Reviewers: Matthias J. Sax <matthias@confluent.io>
Make sure no scaladoc warnings are emitted from the streams-scala project build. We cannot fully fix all scaladoc warnings due to limitations of the scaladoc tool, so this is a best-effort attempt at fixing as many warnings as possible. We also disable one problematic class of scaladoc wornings (link errors) in the gradle build. The causes of existing warnings are that we link to java members from scaladoc, which is not possible, or we fail to disambiguate some members. The broad rule applied in the changes is - For links to Java members such as [[StateStore]], we use the fully qualified name in a code tag to make manual link resolution via a search engine easy. - For some common terms that are also linked to Java members, like [[Serde]], we omit the link. - We disambiguate where possible. - In the special case of @throws declarations with Java Exceptions, we do not seem to be able to avoid the warning altogther. Reviewers: Matthias J. Sax <mjsax@apache.org>, Guozhang Wang <wangguoz@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Committer Checklist (excluded from commit message)