KAFKA-19937: Introduced Shared ReaperThread for Persister / NetworkPartitionMetadataClient#21842
Open
rionmonster wants to merge 4 commits intoapache:trunkfrom
Open
KAFKA-19937: Introduced Shared ReaperThread for Persister / NetworkPartitionMetadataClient#21842rionmonster wants to merge 4 commits intoapache:trunkfrom
rionmonster wants to merge 4 commits intoapache:trunkfrom
Conversation
… within a shared thread
…of Reaper threads
…etworkPartitionMetadataClient
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This pull request addresses the redundant thread usage detailed in KAFKA-19937 affecting the
PersisterStateManagerandNetworkPartitionMetadataClientclasses specifically. Presently each creates/manages its own separateSystemTimerReaperinstances, but rely on identical timers with independent tasks. The changes proposed address this by introducing a new, sharable instance of the thread to reduce overahead.Key Changes
Reaperclass fromSystemTimerReaperinto a new, sharable standaloneReaperThreadclass, which enables shared ownership across multipleSystemTimerReaperinstances.SystemTimerReaperconstructor to support reaper lifecycle management (the existing owning constructor is unchanged).BrokerServerto create a single sharedReaperThreadandSystemTimerinstance used by bothPersisterStateManagerandNetworkPartitionMetadataClient, with cleanup in the shutdown path after both components have been stopped.Tests and Verification
Verified that all existing test suites still pass as expected and added the following to verify new behavior and usage related to the above changes:
ReaperThreadTest.javato verify lifecycle, multiple staggered tasks, and unstarted state.SystemTimerReaperTest.javato verifyclose()behavior and verify partial closures (e.g. closing one of the shared timers still allows other to schedule).Reviewer(s)
Tagging @AndrewJSchofield (initial reporter)