Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reader support seek from separate messageId/time #10348

Merged
merged 11 commits into from
May 15, 2021

Conversation

315157973
Copy link
Contributor

Fixes #9301

Motivation

Currently in ReaderConfigurationData the API allow to ‘setStartMessageId’ only from single message ID and this apply to all consumers in the MultiTopicsReaderImpl.

Is it possible to add start message per partition / topic.

Modifications

Reader can seek by function

Verifying this change

1 For partitioned topic, we can seek by time/messageId for every partition
2 For non-partitioned topic, we can seek by function

@lhotari
Copy link
Member

lhotari commented Apr 25, 2021

/pulsarbot run-failure-checks

@codelipenghui codelipenghui added the doc-required Your PR changes impact docs and you will update later. label Apr 28, 2021
@codelipenghui codelipenghui added this to the 2.8.0 milestone Apr 28, 2021
@Anonymitaet
Copy link
Member

@315157973 thanks for your great work. Could you please help add docs accordingly? Then I can help review, thanks

Copy link
Contributor

@eolivelli eolivelli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lgtm

@sijie
Copy link
Member

sijie commented May 8, 2021

@merlimat @codelipenghui Can you review this PR?

Copy link
Member

@Anonymitaet Anonymitaet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding docs. I’ve left some comments, please check and keep consistencies for all occurrences, thanks.

Besides, docs are added to Reader.java and Consumer.java, Producer.java needs to be updated?

@Anonymitaet Anonymitaet added doc Your PR contains doc changes, no matter whether the changes are in markdown or code files. and removed doc-required Your PR changes impact docs and you will update later. labels May 8, 2021
315157973 and others added 5 commits May 8, 2021 15:24
…onsumer.java

Co-authored-by: Yu Liu <50226895+Anonymitaet@users.noreply.github.com>
…onsumer.java

Co-authored-by: Yu Liu <50226895+Anonymitaet@users.noreply.github.com>
…onsumer.java

Co-authored-by: Yu Liu <50226895+Anonymitaet@users.noreply.github.com>
…onsumer.java

Co-authored-by: Yu Liu <50226895+Anonymitaet@users.noreply.github.com>
@codelipenghui codelipenghui merged commit 7bd849b into apache:master May 15, 2021
@315157973 315157973 deleted the multiReader branch July 19, 2021 11:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
doc Your PR contains doc changes, no matter whether the changes are in markdown or code files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

MultiTopicsReaderImpl – Start from separate MessageId for each topic/ partition
6 participants