[Bug] Pulsar Consumer receives same message more than once in SHARED subscription

[Inbasasi]

Search before asking

• I searched in the issues and found nothing similar.

Version

Apache Pulsar Version - 2.11.1

Minimal reproduce step

The consumer is as below,

pulsarClient.newConsumer(Schema.JSON(Schedule.class)).ackTimeout(600000, TimeUnit.MILLISECONDS) .consumerName("test-consumer").topic(pulsarProperties.getTopic()) .subscriptionName("test-subscription") .subscriptionType(SubscriptionType.Shared) .deadLetterPolicy(DeadLetterPolicy.builder().maxRedeliverCount(5) .deadLetterTopic(pulsarProperties.getTopic() + "-dlq").build()) .autoUpdatePartitions(true).receiverQueueSize(2000) .subscriptionInitialPosition(SubscriptionInitialPosition.Earliest) .messageListener(consumer).subscribe();

What did you expect to see?

The consumer should consume a message only once & that is happening as expected most of the time.
But occasionally, The consumer is consuming the same message twice which is causing consistency issues with the application logic.

What did you see instead?

The consumer is consuming the same message twice which is causing consistency issues with the application logic.

I confirmed the same by logging the message ID in the consumed message.

2023-08-14 02:32:15.221 INFO [1a512eb2-2c23-4b91-bd2e-cc656df3a9c8] [] [pulsar-external-listener-12-1] [RecurringScheduledEventHandler.java:38] c.f.s.s.p.RecurringScheduledEventHandler - >> handling mesage ID 100260:12688:0 - pulsar-105-23

2023-08-14 02:32:15.225 INFO [720cf56f-9b4c-49f7-8f91-81c0fa9bba35] [] [pulsar-external-listener-12-1] [RecurringScheduledEventHandler.java:38] c.f.s.s.p.RecurringScheduledEventHandler - >> handling mesage ID 100260:12688:0 - pulsar-105-23

Here, The same message ID 100260:12688:0 is consumed by the same consumer thread pulsar-external-listener-12-1 within few ms by 1a512eb2-2c23-4b91-bd2e-cc656df3a9c8 & 720cf56f-9b4c-49f7-8f91-81c0fa9bba35

How come the listener pulsar-external-listener-12-1 processes the same messages twice at the same time? The ack is yet to happen for the thread 1a512eb2-2c23-4b91-bd2e-cc656df3a9c8 but before that the same message is consumer & processed by the thread 720cf56f-9b4c-49f7-8f91-81c0fa9bba35

Anything else?

No response

Are you willing to submit a PR?

• I'm willing to submit a PR!

[Inbasasi]

Can someone please help with whether I am making any configuration issues here, which caused the message consumption more than once?

[codelipenghui]

@Inbasasi Without transactions, the shared subscription on a persistent topic can only provide the at-least-once semantic. You can try to check the broker logs, to see if there are other consumers disconnected or if the topic has been unloaded to another broker. Actually, there are many cases to introduce the duplicated messages, then we introduced the transactions to support exactly-once semantic.

[codelipenghui]

@Inbasasi, I'm not sure if the last comment can help you or not. I will close the issue first and feel free to reopen it if needed.