[Issue 11153] [pulsar-broker] Fix for missing metadata-store watch events #11157
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
added 2 commits
June 29, 2021 13:34
- if the cache is updated in admin API handler, the getDataAsync will hit the cache and fails to register a watch for future updates. - fix is to just not update the cache in the admin API handler. This forces the getData APIs to lazily populate the cache via asyncReload, also ensuring that a watch is registered for future updates.
|
@merlimat @jerrypeng @sursingh please help review. |
|
/pulsarbot run-failure-checks |
|
@bharanic-dev I think all these problems of watches not being set would be solved by #11198 |
|
Closing this as, #11198 has been merged. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #11153
Motivation
There is a race condition between the admin API handler updating the cache with the latest changes and the metadata-store get()/getAsync() API that is invoked in the watch callback. If there is a cache hit in the get()/getAsync(), no watch event gets registered for future updates (this likely only affects zookeeper metadata store). This results in watch events getting missed.
Modifications
The admin API handler should not update the cache. This results in the metadata-store get()/getAsync() API lazily populating the cache via asyncReload().
Fix is to not update the cache in admin API handler.
Verifying this change
This change added tests and can be verified as follows:
Added test
org.apache.pulsar.broker.resourcegroup.ResourceGroupConfigListenerTest#testResourceGroupUpdateLoopto verify the changes. The test can be run using mvn command:mvn test -Dtest=org.apache.pulsar.broker.resourcegroup.ResourceGroupConfigListenerTest#testResourceGroupUpdateLoop -pl pulsar-broker
Documentation
this is a bug fix and has no user visible change (except that the issue is fixed). There is no documentation impact.