Introduce messages fence mechanism for Key_Shared subscription

[codelipenghui]

Fixes #6554

Motivation

Introduce messages fence mechanism for Key_Shared subscription to guarantee ordering dispatching.

The fenced message means that the messages can't be delivered at this time. In Key_Shared subscription, the new consumer needs to wait for all messages before the first message that delivers to this consumer are acknowledged. The fencedMessagesContainer maintains all fenced messages by a map (mark-delete-position -> fenced messages). When the mark delete position of the cursor is greater than the mark-delete-position in the fencedMessagesContainer, it means the fenced messages will be lifted.

Modifications

Describe the modifications you've done.

Verifying this change

New unit tests added.

Does this pull request potentially affect one of the following parts:

If yes was chosen, please highlight the changes

• Dependencies (does it add or upgrade a dependency): (no)
• The public API: (no)
• The schema: (no)
• The default values of configurations: (no)
• The wire protocol: (no)
• The rest endpoints: (no)
• The admin cli options: (no)
• Anything that affects deployment: (no)

Documentation

• Does this pull request introduce a new feature? (no)
• If yes, how is the feature documented? (docs)

[codelipenghui]

@merlimat Please help take a look at this PR. This implementation is based on your idea #6554 (comment). Thanks for your great idea.

[Huanli-Meng]

leave my comments, PTAL

[merlimat]

@codelipenghui This is not quite what I was describing in the comment. I don't think it's desirable to "fence" the individual messages, rather just keeping a pointer to where the consumer joined. I have already a fix (working in prod) but it's backed up on #6791.

[codelipenghui]

@merlimat I have added a pointer to indicate where the consumer joined and if the pointer has been mark deleted, the dispatcher dispatch messages to the consumer. The fenced messages just messages buffer, If messages don't belong to this consumer, the messages can continue dispatch to other consumers. If the buffer is full, the dispatcher will stop dispatch messages. I'm not sure if this is a good approach, the main idea does not stop dispatch messages to other consumers.

I have already a fix (working in prod) but it's backed up on #6791.

Is this mean the fix depends on #6791? Or the fix is only for the consistent hash approach.

[codelipenghui]

/pulsarbot run-failure-checks

[Huanli-Meng]

should the key_shared subscription in messaging section be updated?

[feeblefakie]

@codelipenghui
Just a question.
So this and #6791 are solving the same issue with different approaches but the both will be applied ?
#6791 looks like more promising but not fully sure.
Could I get some information about it ?

[codelipenghui]

@feeblefakie #6791 introduces a consistent hash approach for the consumer selector. The current implementation is a hash range split approach.

This PR is to fix #6554

[codelipenghui]

close via #7106