Skip to content

[Issue 242][pulsar-client-go] feature: add send timeout #249

Closed
lifepuzzlefun wants to merge 7 commits intoapache:masterfrom
lifepuzzlefun:master
Closed

[Issue 242][pulsar-client-go] feature: add send timeout #249
lifepuzzlefun wants to merge 7 commits intoapache:masterfrom
lifepuzzlefun:master

Conversation

@lifepuzzlefun
Copy link

@lifepuzzlefun lifepuzzlefun commented May 16, 2020

Fixes #242

Motivation

add send timeout

Modifications

  1. ProduceOptions
    add SendTimeout and SendTimeoutCheckInterval in ProducerOptions

  2. BlockingQueue
    unexported BlockingQueue.Iterator
    and add methods to avoid race condition.
    BlockingQueue.IterateIfNonEmpty and BlockingQueue.PollIfSatisfy

  3. partitionProducer
    add a ticker to check if pendingItem in pendingQueue has expired.

Verifying this change

  • Make sure that the change passes the CI checks.

This change added tests and can be verified as follows:

add some tiny test in producer_test.go

Does this pull request potentially affect one of the following parts:

If yes was chosen, please highlight the changes

  • Dependencies (does it add or upgrade a dependency): no
  • The public API: no
  • The schema: no
  • The default values of configurations: no
  • The wire protocol: no

Documentation

  • Does this pull request introduce a new feature? yes
  • If yes, how is the feature documented? (not applicable / docs / GoDocs / not documented)
  • If a feature is not applicable for documentation, explain why?
  • If a feature is not documented yet in this PR, please create a followup issue for adding the documentation

Changes

1. ProduceOptions
add SendTimeout and SendTimeoutCheckInterval in ProducerOptions

2. BlockingQueue
unexported BlockingQueue.Iterator
and add methods to avoid race condition.
BlockingQueue.IterateIfNonEmpty and BlockingQueue.PollIfSatisfy

3. partitionProducer
add a ticker to check if pendingItem in pendingQueue has expired.

Some explain on implementation:

1. if message is send as batch, will only check if the first message timeout.
   if first message in batch timeout. will call all message send callback with an ErrSendTimeout.

2. avoid use context to judge if send has been timeout.
   maybe one set different timeout on each message. this will introduce a lot complicity.
   and context is use for goroutine control. i think this is not suit for send timeout use case, because there is no goroutine at all.
Changes

1. ProduceOptions
add SendTimeout and SendTimeoutCheckInterval in ProducerOptions

2. BlockingQueue
unexported BlockingQueue.Iterator
and add methods to avoid race condition.
BlockingQueue.IterateIfNonEmpty and BlockingQueue.PollIfSatisfy

3. partitionProducer
add a ticker to check if pendingItem in pendingQueue has expired.

Some explain on implementation:

1. if message is send as batch, will only check if the first message timeout.
   if first message in batch timeout. will call all message send callback with an ErrSendTimeout.

2. avoid use context to judge if send has been timeout.
   maybe one set different timeout on each message. this will introduce a lot complicity.
   and context is use for goroutine control. i think this is not suit for send timeout use case, because there is no goroutine at all.
# Conflicts:
#	pulsar/producer.go
#	pulsar/producer_partition.go
# Conflicts:
#	pulsar/producer.go
#	pulsar/producer_partition.go
@lifepuzzlefun lifepuzzlefun changed the title feature: add send timeout (#242) [Issue 242][component]feature: add send timeout (#242) May 16, 2020
@lifepuzzlefun lifepuzzlefun changed the title [Issue 242][component]feature: add send timeout (#242) [Issue 242][pulsar-client-go] feature: add send timeout May 16, 2020
@merlimat
Copy link
Contributor

The Send() is already taking a Context object. If the context has a timeout, it will be used.

@merlimat
Copy link
Contributor

I think the correct fix for #242 would be to also check the context before attempting to publish the message.

@lifepuzzlefun
Copy link
Author

yeah, i think it should be, but when user pass different context with different sendtimeout to Send and SendAsync , this will introduce a lot of complicity. the message is send as batch. and each message in batch with different timeout

@lifepuzzlefun
Copy link
Author

lifepuzzlefun commented May 16, 2020

i'm confused about the above use case, need some guide on the real popuse of passing an context here. and where do you prefer to check the context? just before send or both before send and time
wait for response.

@yarthur1
Copy link
Contributor

I hope that both before send and time wait for response should check timeout.I don't want to wait send func always,because I meet this situation where pulsar broker has something wrong and send func always wait @

@lifepuzzlefun
Copy link
Author

closed and goto #252

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

send add timeout

3 participants

Comments