Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[C++] Fix message id is always the default value in send callback #6812

Merged
merged 4 commits into from
Apr 25, 2020

Conversation

BewareMyPower
Copy link
Contributor

Motivation

After commit of #4817, the send callback's 2nd argument became MessageId, but the MessageId in callback is always the default value (-1, -1, -1, -1). We need the message id in send callback if messages were sent successfully.

The problem is that the correct message id has been retrieved in ProducerImpl::ackReceived but not passed to the user provided callback. Because after messages were sent to BatchMessageContainer, the wrapper of user provided callback used the MessageId of user constructed Message as the 2nd argument, which is always (-1, -1, -1, -1).

Modifications

  • Remove useless field messageId of BatchMessageContainer::MessageContainer. Then add const MessageId& argument to batchMessageCallBack instead.
  • Add tests for message id in send callback. Specially, for a batched message, each internal message's batch index was verified.

Verifying this change

  • Make sure that the change passes the CI checks.

This change added tests and can be verified as follows:

Run BasicEndToEndTest.testSendCallback and BatchMessageTest.testSendCallback.

@BewareMyPower
Copy link
Contributor Author

/pulsarbot run cpp-tests

@BewareMyPower
Copy link
Contributor Author

/pulsarbot run unit-tests

ASSERT_EQ(ResultOk, result);
ASSERT_EQ(i, id.batchIndex());
sentIdSet.emplace(id);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's no hard guarantee that this callback will be executed before the receive loop is done.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for pointing out this! I missed the point that the SendReceipt to producer is not guaranteed to be completed before the consumer's receive() done. I'll do some work on it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a Latch class in lib/ that might be helpful here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! I found the Latch just now :)

@merlimat merlimat added the type/bug The PR fixed a bug or issue reported a bug label Apr 24, 2020
@merlimat merlimat added this to the 2.6.0 milestone Apr 24, 2020
@BewareMyPower
Copy link
Contributor Author

/pulsarbot run-failure-checks

1 similar comment
@BewareMyPower
Copy link
Contributor Author

/pulsarbot run-failure-checks

@merlimat merlimat merged commit fc69a62 into apache:master Apr 25, 2020
gaoran10 pushed a commit to gaoran10/pulsar that referenced this pull request May 5, 2020
…ache#6812)

* Fix bug: sendCallback's 2nd argument was always the default MessageId

* Set batch index for each message's callback of batch

* Add test for message id in send callback

* Ensure all send callbacks completed before ASSERT_EQ
gaoran10 pushed a commit to gaoran10/pulsar that referenced this pull request May 5, 2020
…ache#6812)

* Fix bug: sendCallback's 2nd argument was always the default MessageId

* Set batch index for each message's callback of batch

* Add test for message id in send callback

* Ensure all send callbacks completed before ASSERT_EQ
addisonj pushed a commit to instructure/pulsar that referenced this pull request May 7, 2020
…ache#6812)

* Fix bug: sendCallback's 2nd argument was always the default MessageId

* Set batch index for each message's callback of batch

* Add test for message id in send callback

* Ensure all send callbacks completed before ASSERT_EQ
jiazhai pushed a commit that referenced this pull request May 8, 2020
)

* Fix bug: sendCallback's 2nd argument was always the default MessageId

* Set batch index for each message's callback of batch

* Add test for message id in send callback

* Ensure all send callbacks completed before ASSERT_EQ
(cherry picked from commit fc69a62)
@BewareMyPower BewareMyPower deleted the send-cb-dev branch May 11, 2020 02:01
Huanli-Meng pushed a commit to Huanli-Meng/pulsar that referenced this pull request May 27, 2020
…ache#6812)

* Fix bug: sendCallback's 2nd argument was always the default MessageId

* Set batch index for each message's callback of batch

* Add test for message id in send callback

* Ensure all send callbacks completed before ASSERT_EQ
huangdx0726 pushed a commit to huangdx0726/pulsar that referenced this pull request Aug 24, 2020
…ache#6812)

* Fix bug: sendCallback's 2nd argument was always the default MessageId

* Set batch index for each message's callback of batch

* Add test for message id in send callback

* Ensure all send callbacks completed before ASSERT_EQ
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release/2.5.2 type/bug The PR fixed a bug or issue reported a bug
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants