Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[improve][doc] clarify detailed behaviors for failover subscription type #633

Merged
merged 2 commits into from Jul 11, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
45 changes: 35 additions & 10 deletions docs/concepts-messaging.md
Expand Up @@ -521,6 +521,12 @@ A master consumer is picked for a non-partitioned topic or each partition of a p

When the master consumer disconnects, all (non-acknowledged and subsequent) messages are delivered to the next consumer in line.

::: note

In some cases, a partition may have an older active consumer processing messages while a newly switched over active consumer starts receiving new messages. This may lead to message duplication or out-of-order.

:::

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this note is not rendering correctly on the website

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch! I'll fix it. Thanks!

##### Failover | Partitioned topics

For partitioned topics, the broker sorts consumers by priority and lexicographical order of consumer name.
Expand All @@ -533,25 +539,32 @@ A consumer is selected by running a module operation `mod (partition index, cons

For example, in the diagram below, this partitioned topic has 2 partitions and there are 4 consumers.

Each partition has 1 active consumer and 1 stand-by consumer.
Each partition has 1 active consumer and 3 stand-by consumers.

- For p0, consumer A is the master consumer, while consumer B would be the next consumer in line to receive messages if consumer A is disconnected.
- For P0, Consumer A is the master consumer, while Consumer B, Consumer C, and Consumer D would be the next consumer in line to receive messages if consumer A is disconnected.

- For p1, consumer C is the master consumer, while consumer D would be the next consumer in line to receive messages if consumer C is disconnected.
- For P1, Consumer B is the master consumer, while Consumer A, Consumer C, and Consumer D would be the next consumer in line to receive messages if consumer B is disconnected.

![Failover subscriptions](/assets/pulsar-failover-subscriptions-4.svg)
- Moreover, if Consumer A and consumer B are disconnected, then

- for P0: Consumer C is the active consumer and Consumer D is the stand-by consumer.

- for P1: Consumer D is the active consumer and Consumer C is the stand-by consumer.

![Failover subscriptions](/assets/pulsar-failover-subscriptions-5.png)

- If the number of partitions in a partitioned topic is **greater** than the number of consumers:

For example, in the diagram below, this partitioned topic has 9 partitions and 3 consumers.

- p0, p3, and p6 are assigned to consumer A.
- P0, P3, and P6 are assigned to Consumer A. Consumer A is their active consumer. Consumer B and Consumer C are their stand-by consumers.

- p1, p4, and p7 are assigned to consumer B.
- P1, P4, and P7 are assigned to Consumer B. Consumer B is their active consumer. Consumer A and Consumer C are their stand-by consumers.

- p2, p5, and p8 are assigned to consumer C.
- P2, P5, and P8 are assigned to Consumer C. Consumer C is their active consumer. Consumer A and Consumer B are their stand-by consumers.

![Failover subscriptions](/assets/pulsar-failover-subscriptions-1.svg)

##### Failover | Non-partitioned topics

- If there is one non-partitioned topic. The broker picks consumers in the order they subscribe to non-partitioned topics.
Expand All @@ -568,9 +581,9 @@ A consumer is selected by running a module operation `mod (partition index, cons

For example, in the diagram below, there are 4 non-partitioned topics and 2 consumers.

- The non-partitioned topic 1 and non-partitioned topic 4 are assigned to consumer B.
- The non-partitioned topic 1 and non-partitioned topic 4 are assigned to consumer B. Consumer A is their stand-by consumer.

- The non-partitioned topic 2 and non-partitioned topic 3 are assigned to consumer A.
- The non-partitioned topic 2 and non-partitioned topic 3 are assigned to consumer A. Consumer B is their stand-by consumer.

![Failover subscriptions](/assets/pulsar-failover-subscriptions-3.svg)

Expand All @@ -594,6 +607,18 @@ In the *Key_Shared* type, multiple consumers can attach to the same subscription

![Key_Shared subscriptions](/assets/pulsar-key-shared-subscriptions.svg)

:::note

If there is a newly switched over active consumer, it will start reading messages from the position where messages are acked by the old inactive consumer.

For example, if P0 is assigned to Consumer A. Consumer A is the active consumer and Consumer B is the stand-by consumer.

- If Consumer A gets disconnected without reading any messages from P0, when Consumer C is added and becomes the new active consumer, then Consumer C will start reading messages directly from P0.

- If Consumer A gets disconnected after reading messages (0,1,2,3) from P0, when Consumer C is added and becomes the active consumer, then Consumer C will start reading messages (4,5,6,7) from P0.

:::

There are three types of mapping algorithms dictating how to select a consumer for a given message key (or ordering key): Sticky, Auto-split Hash Range, and Auto-split Consistent Hashing. The steps for all algorithms are:
1. The message key (or ordering key) is passed to a hash function (e.g., Murmur3 32-bit), yielding a 32-bit integer hash.
2. That hash number is fed to the algorithm to select a consumer from the existing connected consumers.
Expand Down Expand Up @@ -652,7 +677,7 @@ C1 is disconnected:
|------- C3 ------|-------------------------- C2 -----------------------|
```

The advantages of this algorithm is that it affects only a single existing consumer upon add/delete consumer, at the expense of regions not evenly sized. Thi means some consumers gets more keys that others. The next algorithm does the other way around.
The advantages of this algorithm is that it affects only a single existing consumer upon add/delete consumer, at the expense of regions not evenly sized. This means some consumers gets more keys that others. The next algorithm does the other way around.

##### Auto-split Consistent Hashing

Expand Down
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Known issue 1: this image style (drawn in Figma) does not match the style of other images (drawn in LucidChart) because we can't log in LucidChart suddenly. We have contacted the support team and we're waiting for feedback. Not sure how long it takes.

Will align the image style once we can log in LucidChart.

Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.