Skip to content

KAFKA-19937: Introduced Shared ReaperThread for Persister / NetworkPartitionMetadataClient#21842

Open
rionmonster wants to merge 4 commits intoapache:trunkfrom
rionmonster:KAFKA-19937
Open

KAFKA-19937: Introduced Shared ReaperThread for Persister / NetworkPartitionMetadataClient#21842
rionmonster wants to merge 4 commits intoapache:trunkfrom
rionmonster:KAFKA-19937

Conversation

@rionmonster
Copy link
Contributor

@rionmonster rionmonster commented Mar 20, 2026

Description

This pull request addresses the redundant thread usage detailed in KAFKA-19937 affecting the PersisterStateManager and NetworkPartitionMetadataClient classes specifically. Presently each creates/manages its own separate SystemTimerReaper instances, but rely on identical timers with independent tasks. The changes proposed address this by introducing a new, sharable instance of the thread to reduce overahead.

Key Changes

  • Extracted the inner Reaper class from SystemTimerReaper into a new, sharable standalone ReaperThread class, which enables shared ownership across multiple SystemTimerReaper instances.
  • Created a new, separate SystemTimerReaper constructor to support reaper lifecycle management (the existing owning constructor is unchanged).
  • Updated BrokerServer to create a single shared ReaperThread and SystemTimer instance used by both PersisterStateManager and NetworkPartitionMetadataClient, with cleanup in the shutdown path after both components have been stopped.

Tests and Verification

Verified that all existing test suites still pass as expected and added the following to verify new behavior and usage related to the above changes:

  • Added new ReaperThreadTest.java to verify lifecycle, multiple staggered tasks, and unstarted state.
  • Extended SystemTimerReaperTest.java to verify close() behavior and verify partial closures (e.g. closing one of the shared timers still allows other to schedule).

Reviewer(s)

Tagging @AndrewJSchofield (initial reporter)

@github-actions github-actions bot added triage PRs from the community core Kafka Broker labels Mar 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core Kafka Broker triage PRs from the community

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant