Skip to content

Create durable exchange topic cursor for queue#619

Merged
codelipenghui merged 9 commits intostreamnative:masterfrom
gaoran10:async-open-exchange-cursor
Oct 10, 2022
Merged

Create durable exchange topic cursor for queue#619
codelipenghui merged 9 commits intostreamnative:masterfrom
gaoran10:async-open-exchange-cursor

Conversation

@gaoran10
Copy link
Copy Markdown
Collaborator

@gaoran10 gaoran10 commented Jul 24, 2022

Motivation

Currently, for AoP, messages of the queue topic are only index data, the original data are in the exchange topic, the AmqpConsumer needs to get the original message data from the exchange topic and send it to the client, if messages in the exchange topic are clean up, the AmqpConsumer will not get the data, so queue topics need durable cursors of exchange topic to make sure the original data exists.

Notable

This PR will cause the exchange topic can't be cleaned up because the queue only handles part of the exchange data, only these index positions could be acknowledged. I'll create a new PR to fix this problem.

Modifications

Create durable cursors of the exchange topic for each queue in an async way.

Does this pull request potentially affect one of the following parts:

If yes was chosen, please highlight the changes

  • Dependencies (does it add or upgrade a dependency): (no)
  • The public API: (no)
  • The schema: (no)
  • The default values of configurations: (no)
  • The wire protocol: (no)
  • The rest endpoints: (no)
  • The admin cli options: (no)
  • Anything that affects deployment: (no)

Documentation

Check the box below.

Need to update docs?

  • doc-required

    (If you need help on updating docs, create a doc issue)

  • no-need-doc

    (Please explain why)

  • doc

    (If this PR contains doc changes)

@gaoran10 gaoran10 requested review from a team and codelipenghui as code owners July 24, 2022 07:35
@github-actions github-actions Bot added the no-need-doc This pr does not need any document label Jul 24, 2022
@gaoran10 gaoran10 changed the title Async open exchange cursor Create durable cursors for the exchange topic Aug 22, 2022
Copy link
Copy Markdown
Contributor

@GhostBoyBoy GhostBoyBoy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@gaoran10 gaoran10 force-pushed the async-open-exchange-cursor branch from 575320c to 01a0109 Compare October 3, 2022 02:08
@gaoran10 gaoran10 changed the title Create durable cursors for the exchange topic Create durable exchange topic cursor for queue Oct 3, 2022
@codelipenghui codelipenghui merged commit ac76399 into streamnative:master Oct 10, 2022
@codelipenghui codelipenghui deleted the async-open-exchange-cursor branch October 10, 2022 00:58
@gaoran10
Copy link
Copy Markdown
Collaborator Author

I create a PR #698 to fix the exchange data cleanup problem.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

no-need-doc This pr does not need any document release/2.10.x

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants