Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

In Pulsar client, the effective value of NackRedeliveryDelay is about 1/3 of its specified value #5344

Open
Jesse-Zhang-Bose opened this issue Oct 9, 2019 · 6 comments

Comments

@Jesse-Zhang-Bose
Copy link

@Jesse-Zhang-Bose Jesse-Zhang-Bose commented Oct 9, 2019

Describe the bug
the effective value of NackRedeliveryDelay is about 1/3 of its specified value.

To Reproduce
Steps to reproduce the behavior:
In my test, I set NackRedeliveryDelay=200 second delay, but it is retrying at second 57
See my log:

2 - ID: (10,1,-1,-1)
3 - ID: (10,2,-1,-1)
4 - ID: (10,3,-1,-1)
5 - ID: (10,4,-1,-1)
6 - ID: (10,5,-1,-1)
7 - ID: (10,6,-1,-1)
8 - ID: (10,7,-1,-1)
9 - ID: (10,8,-1,-1)
10 - ID: (10,9,-1,-1)   //at second 10,  processed message #9
68 - ID: (10,9,-1,-1)    //at second 68,  processed message #9 again,  delay is 57 seconds only.
69 - ID: (10,0,-1,-1)    //at second 69,  processed message #0 again,  delay is 68 seconds only.

Expected behavior
The delay for the nack redelivery is not meant to be precise, but it should not be around 1/3 of the value. In my case, retrying should happen around 200s

Desktop (please complete the following information):
Mac, or linux

Additional context
-Standalone Pular server 2.4.1. or 2.3.1
-Python or golang, with C++ client 2.4.0

@zymap

This comment has been minimized.

Copy link
Contributor

@zymap zymap commented Nov 4, 2019

Hi @Jesse-Zhang-Bose . I think that is the right process.

There are two variables of NackRedelivery. One is NackRedeliveryDelay, another is TimeInterval.

The NackRedeliverDelay is the message should redelivery after how long. And the TimeInterval is the scheduler time interval.

The scheduler will check is there any messages that need to redeliver. And it will be scheduled with the time gap specified by TimeInterval. The TimeInterval is set to 1/3 NackRedeliveryDelay. So you will see it retry after 1/3 of nack redelivery delay.

You can see the implementation of the Nack at here https://github.com/apache/pulsar/blob/master/pulsar-client-cpp/lib/NegativeAcksTracker.cc

It implemented as same as java

private synchronized void triggerRedelivery(Timeout t) {

@Jesse-Zhang-Bose

This comment has been minimized.

Copy link
Author

@Jesse-Zhang-Bose Jesse-Zhang-Bose commented Nov 4, 2019

@zymap, thanks for the feedback.
The internal work does explain the behavior I reported. But I would still think the behavior itself is unexpected, or at least quite confusing.

@zymap

This comment has been minimized.

Copy link
Contributor

@zymap zymap commented Nov 5, 2019

@Jesse-Zhang-Bose

I think we can expose the TimeInterval to config. What do you think?

Because of all nacked messages will queue into the redeliver map. We need to keep the TimeInterval as small as possible so that we can redeliver the nacked message quicker.

As your case, if it retries after 200s, it will redeliver the first message and the rest messages will redeliver after the next 200s if the messages are not sent at the same time as the first message.

@zymap

This comment has been minimized.

Copy link
Contributor

@zymap zymap commented Dec 3, 2019

Hi @Jesse-Zhang-Bose, I think exposing the TimeInterval might is not a good idea to address this.

I will close this issue if you have not any problem.

@Jesse-Zhang-Bose

This comment has been minimized.

Copy link
Author

@Jesse-Zhang-Bose Jesse-Zhang-Bose commented Dec 6, 2019

@zymap i agree with you,timeInterval is pulsar's internal work, exposing it is not necessary. However, I would still think this is an issue need to fix, maybe use another way?

@zymap

This comment has been minimized.

Copy link
Contributor

@zymap zymap commented Dec 19, 2019

@Jesse-Zhang-Bose Sorry for my late response.

I think we can use the NackRedeliveryDelay as the redeliver time. So you can configure it by yourself.

Hi @merlimat. Because you wrote the NackRedelivery, what do you think? Can we use NackRedeliveryDelay?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.