Skip to content

[improve][broker]Reduce the frequency of calls to the LockManager#listLocks method.#17809

Closed
lordcheng10 wants to merge 2 commits intoapache:masterfrom
lordcheng10:reduce_list_brokers_count
Closed

[improve][broker]Reduce the frequency of calls to the LockManager#listLocks method.#17809
lordcheng10 wants to merge 2 commits intoapache:masterfrom
lordcheng10:reduce_list_brokers_count

Conversation

@lordcheng10
Copy link
Contributor

@lordcheng10 lordcheng10 commented Sep 23, 2022

Motivation

In the cleanupDeadBrokersData method, the ModularLoadManagerImpl#getAvailableBrokers method will be called every time, in which LockManager#listLocks will be called:

private void cleanupDeadBrokersData() {
final Set<String> activeBrokers = getAvailableBrokers();
final Set<String> knownBrokersCopy = new HashSet<>(this.knownBrokers);
Collection<String> newBrokers = CollectionUtils.subtract(activeBrokers, knownBrokersCopy);
this.knownBrokers.addAll(newBrokers);
Collection<String> deadBrokers = CollectionUtils.subtract(knownBrokersCopy, activeBrokers);
this.knownBrokers.removeAll(deadBrokers);
if (pulsar.getLeaderElectionService() != null
&& pulsar.getLeaderElectionService().isLeader()) {
deadBrokers.forEach(this::deleteTimeAverageDataFromMetadataStoreAsync);
}
}

For non-leader nodes, it can return directly without executing the listLocks method.

Documentation

  • doc-required
    (Your PR needs to update docs and you will update later)

  • doc-not-needed
    (Please explain why)

  • doc
    (Your PR contains doc changes)

  • doc-complete
    (Docs have been already added)

Matching PR in forked repository

PR in forked repository: <!-- ENTER URL HERE

After opening this PR, the build in apache/pulsar will fail and instructions will
be provided for opening a PR in the PR author's forked repository.

apache/pulsar pull requests should be first tested in your own fork since the
apache/pulsar CI based on GitHub Actions has constrained resources and quota.
GitHub Actions provides separate quota for pull requests that are executed in
a forked repository.

The tests will be run in the forked repository until all PR review comments have
been handled, the tests pass and the PR is approved by a reviewer.

-->

@github-actions github-actions bot added the doc-not-needed Your PR changes do not impact docs label Sep 23, 2022
@lordcheng10 lordcheng10 changed the title [fix][broker]Reduce the frequency of calls to the LockManager#listLocks method. [improve][broker]Reduce the frequency of calls to the LockManager#listLocks method. Sep 23, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

doc-not-needed Your PR changes do not impact docs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant