Use historical data from old API in new API #329
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
merlimat
reviewed
Apr 4, 2017
| if (zkClient.exists(bundleZPath, null) != null) { | ||
| bundleData = readJson(zkClient.getData(bundleZPath, null, null), BundleData.class); | ||
| } else if (zkClient.exists(quotaZPath, null) != null) { |
There was a problem hiding this comment.
Both the zkClient.exists() calls are blocking here (it needs to send a req to zk server and wait for it). You can leverage the ZK cache classes that are already in place to cache the existence check for that znode and use watches to update the status.
There was a problem hiding this comment.
For the leader broker, these calls will only be made once per bundle, and then they are added to the local bundle data map -- will it still help to use a cache in this case?
merlimat
added
the
type/enhancement
rdhabalia
pushed a commit
that referenced
this pull request
May 2, 2017
hrsakai
pushed a commit
to hrsakai/pulsar
that referenced
this pull request
Dec 10, 2020
…che#329) Signed-off-by: Paulo Pereira <paulo.pereira@karhoo.com> ### Motivation I have a service that when it restarts, it needs to know what was the last message successfully sent to pulsar. A reader seems the logical place, since we can specify `StartMessageID` as `LatestMessageID()` and `StartMessageIDInclusive` ### Modifications When the reader is created, verify if it startMessageIDInclusive true and startMessageID == lastestMessageID() and then get the last message id and seek to that message id.
hangc0276
pushed a commit
to hangc0276/pulsar
that referenced
this pull request
May 26, 2021
fix apache#329 fix highwatermark and logendoffset error
5 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
Currently, users which switch to
ModularLoadManagerImplwill carry over no history contained in the resource quotas, causing the default message rate and throughput values to be used for every bundle. This PR makes it so thatModularLoadManagerImplwill use this data to initialize bundle data for bundles not yet handled by the new API.Modifications
When assigning new bundles, the historical data will now be initialized according to the following priorities:
/loadbalance/bundle-data/loadbalance/resource-quota/namespaceResult
We may now utilize pre-existing historical data for
ModularLoadManagerImpl.