Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[pulsar-client] Add conf backoff values #12520

Merged

Conversation

callumduffy
Copy link
Contributor

Motivation

Allowing services waiting on pulsar container etc to use the backoff values given in the config when the client is initialised. Using a service where i handle the future on my side, but i feel this should be in the service as is!

Modifications

Edit the values used for the backoff to use the client values.
Defaults are the exact same as the values used previous to this change.

Verifying this change

  • Make sure that the change passes the CI checks.

This change is a trivial rework / code cleanup without any test coverage.

Does this pull request potentially affect one of the following parts:

If yes was chosen, please highlight the changes

  • The public API: (yes) - slight change here for anyone who has set values for backoff, timing on retries will use these values.

Documentation

Need to update docs?

  • doc-required

    (If you need help on updating docs, create a doc issue)

  • no-need-doc

    (Please explain why)

  • doc

    (If this PR contains doc changes)

@github-actions github-actions bot added the doc-not-needed Your PR changes do not impact docs label Oct 28, 2021
@@ -884,9 +884,9 @@ public void reloadLookUp() throws PulsarClientException {
TopicName topicName = TopicName.get(topic);
AtomicLong opTimeoutMs = new AtomicLong(conf.getLookupTimeoutMs());
Backoff backoff = new BackoffBuilder()
.setInitialTime(100, TimeUnit.MILLISECONDS)
.setInitialTime(conf.getInitialBackoffIntervalNanos, TimeUnit.NANOSECONDS)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice catch, looks like we apply the configuration for producer and consumer but missed the client part.

Could you please help add a test for covering the changes to make sure we will not make a regression in the future?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@codelipenghui - what would you like to see tested? I think it is pretty well defined to use the config for the backoff here. My only concern would be backwards compatibility, but I don't see that being a real issue. The default config here is 100 milliseconds.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@michaeljmarshall I mean to add a test to cover the backoff interval is configurable.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@codelipenghui - thanks for clarifying. I'm not exactly sure how to add that test, but I think we could probably merge this as is.

@codelipenghui codelipenghui added release/2.8.3 release/2.9.1 type/bug The PR fixed a bug or issue reported a bug labels Nov 6, 2021
@codelipenghui codelipenghui added this to the 2.10.0 milestone Nov 6, 2021
Copy link
Member

@michaeljmarshall michaeljmarshall left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@callumduffy - thanks for your contribution. I noticed that ConsumerImpl#internalGetLastMessageIdAsync() also has the value for initialTime hard coded. Would you like to update that configuration as well?

@@ -884,9 +884,9 @@ public void reloadLookUp() throws PulsarClientException {
TopicName topicName = TopicName.get(topic);
AtomicLong opTimeoutMs = new AtomicLong(conf.getLookupTimeoutMs());
Backoff backoff = new BackoffBuilder()
.setInitialTime(100, TimeUnit.MILLISECONDS)
.setInitialTime(conf.getInitialBackoffIntervalNanos, TimeUnit.NANOSECONDS)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@codelipenghui - what would you like to see tested? I think it is pretty well defined to use the config for the backoff here. My only concern would be backwards compatibility, but I don't see that being a real issue. The default config here is 100 milliseconds.

@michaeljmarshall michaeljmarshall merged commit 970363c into apache:master Feb 11, 2022
@michaeljmarshall michaeljmarshall modified the milestones: 2.11.0, 2.10.0 Feb 11, 2022
michaeljmarshall pushed a commit that referenced this pull request Feb 11, 2022
* add conf backoff values

* Apply suggestions from code review

Co-authored-by: Callum Duffy <callum.duffy@toasttab.com>
Co-authored-by: Michael Marshall <mikemarsh17@gmail.com>

### Motivation

Allowing services waiting on pulsar container etc to use the backoff values given in the config when the client is initialised. Using a service where i handle the future on my side, but i feel this should be in the service as is!

### Modifications

Edit the values used for the backoff to use the client values.
Defaults are the exact same as the values used previous to this change.

(cherry picked from commit 970363c)
@michaeljmarshall michaeljmarshall added the cherry-picked/branch-2.8 Archived: 2.8 is end of life label Feb 11, 2022
codelipenghui pushed a commit that referenced this pull request Feb 16, 2022
* add conf backoff values

* Apply suggestions from code review

Co-authored-by: Callum Duffy <callum.duffy@toasttab.com>
Co-authored-by: Michael Marshall <mikemarsh17@gmail.com>

### Motivation

Allowing services waiting on pulsar container etc to use the backoff values given in the config when the client is initialised. Using a service where i handle the future on my side, but i feel this should be in the service as is!

### Modifications

Edit the values used for the backoff to use the client values.
Defaults are the exact same as the values used previous to this change.

(cherry picked from commit 970363c)
@codelipenghui codelipenghui added the cherry-picked/branch-2.9 Archived: 2.9 is end of life label Feb 16, 2022
Nicklee007 pushed a commit to Nicklee007/pulsar that referenced this pull request Apr 20, 2022
* add conf backoff values

* Apply suggestions from code review

Co-authored-by: Callum Duffy <callum.duffy@toasttab.com>
Co-authored-by: Michael Marshall <mikemarsh17@gmail.com>

### Motivation

Allowing services waiting on pulsar container etc to use the backoff values given in the config when the client is initialised. Using a service where i handle the future on my side, but i feel this should be in the service as is!

### Modifications

Edit the values used for the backoff to use the client values.
Defaults are the exact same as the values used previous to this change.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/client cherry-picked/branch-2.8 Archived: 2.8 is end of life cherry-picked/branch-2.9 Archived: 2.9 is end of life doc-not-needed Your PR changes do not impact docs release/2.8.3 release/2.9.2 type/bug The PR fixed a bug or issue reported a bug
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants