[improve][pip] PIP-332: peek messages from topic subscription with messagePosition value#22304
Open
shravannarayan-es wants to merge 2540 commits intoapache:branch-2.11from
Open
[improve][pip] PIP-332: peek messages from topic subscription with messagePosition value#22304shravannarayan-es wants to merge 2540 commits intoapache:branch-2.11from
shravannarayan-es wants to merge 2540 commits intoapache:branch-2.11from
Conversation
…ead compact mode disable (apache#21969)
…lsar docker images (apache#21872)
…Doc (apache#21980) Co-authored-by: Matteo Merli <mmerli@apache.org>
…22022) ### Motivation For some use case, the users need to store all the messages even though these message are acked by all subscription. So they set the retention policy of the namespace to infinite retention (setting both time and size limits to `-1`). But the data in the system topic does not need for infinite retention. ### Modifications For system topics, do not retain messages that have already been acknowledged.
…addTransaction (apache#22020) ### Motivation According to the method signature, we should return true when the transaction is added to the tracker. ``` /** * Add a txnID to the tracker. * * @param sequenceId * the sequenceId * @param timeout * the absolute timestamp for transaction timeout * * @return true if the transaction was added to the tracker or false if had timed out */ CompletableFuture<Boolean> addTransaction(long sequenceId, long timeout); ``` But actually, we return false for any cases. Update: Moreover, we do not use the return value anyway, it is useless. We have better remove it. ### Modifications ~~Return true when the transaction is added to the tracker.~~ Remove the return value as it is useless.
Co-authored-by: Zac Bentley <zbentley@users.noreply.github.com>
Co-authored-by: Matteo Merli <mmerli@apache.org>
…-io/solr (apache#22047) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
- https://learn.microsoft.com/en-us/azure/data-explorer/ Co-authored-by: Ramachandran A G <106139410+ag-ramachandran@users.noreply.github.com>
…rting broker registry. (apache#22065)
…-404` when calling `Admin API` and the topic does not exist. (apache#21995)
… in expiryMon… (apache#21991) Co-authored-by: atomchchen <atomchchen@tencent.com>
…e topic level replication (apache#22537)
… an orphan replicator in the previous topic owner (apache#21948)
Signed-off-by: Zixuan Liu <nodeces@gmail.com>
…etrics gzip compression (apache#22576)
…ent topics in SHARED subscriptions (apache#22536)
… replicateSubscriptionState (apache#22572)
…oad non-persistent topic fails and fix the flaky test testBrokerStatsTopicLoadFailed (apache#22580)
shravannarayan-es
force-pushed
the
peekWithPosition-v2.11
branch
from
16982cb to
340d60d
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PIP: 332
Motivation
Currently we are able to peek any number of messages of a topic subscription backlog using the Pulsar Java Admin API.
However, we are only able to view these messages starting from the most recent message i.e messagePosition of 1. If we want to view the 100th message, we have to peek all the top 100 messages.
When the number of messages is large, we want to split them by paging them and displaying them on the UI. So, we would like the ability to view any batch of messages instead of all of the top messages. With the messagePosition and numberOfMessages that should be fairly straightforward to do.
Modifications
The Pulsar Admin API in Java currently provides a way to peek messages using this
This function includes a default hardcoded messagePosition value of 1. We want to give the user the option to input a different messagePosition value.
The current peekMessages(String topic, String subName, int numMessages) method will remain the same and use messagePosition of 1.
We will be overloading this method with messagePosition. It will default to 1 if not specified.
Verifying this change
This change is a trivial rework / code cleanup without any test coverage.
Does this pull request potentially affect one of the following parts:
If the box was checked, please highlight the changes
Documentation
docdoc-requireddoc-not-neededdoc-completePR for pulsar-site doc updates: [improve][site] Doc changes to support peekMessages with Offset in admin-api-topics pulsar-site#840
Matching PR in forked repository
PR in forked repository: