Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[refactor]: Refactor the toTrackingMessageID() #972

Merged
merged 2 commits into from
Mar 7, 2023

Conversation

Gleiphir2769
Copy link
Contributor

@Gleiphir2769 Gleiphir2769 commented Feb 28, 2023

Motivation

toTrackingMessageID() is a function that is widely used in consumer implementation. It can convert an interface type MessageID into an struct type trackingMessageID. In addition, the second return value also plays a role in checking the MessageID type. In other words, it indicates that MessageID cannot be a user-defined type. From the perspective of code readability, toTrackingMessageID() should not do both.

Note: After #968 , toTrackingMessageID() returns only a pointer now. The role of original ok is replaced by nil pointer now. However, the main content discussed in this PR has not changed.

For example.

func (c *regexConsumer) AckID(msgID MessageID) error {
mid, ok := toTrackingMessageID(msgID)
if !ok {
c.log.Warnf("invalid message id type %T", msgID)
return errors.New("invalid message id type")
}

This example is the correct usage. The ok returned by toTrackingMessageID() is used to reject user-defined MessageID.

trackingID, ok := toTrackingMessageID(msgID)
if !ok {
return errors.New("failed to convert trackingMessageID")
}

This example is a bit vague. The actual effect here is the same as the previous example. But it return an error failed to convert trackingMessageID which is confusing.

for _, mid := range c.chunkedMsgIDs {
pc.log.Info("Removing chunk message-id", mid.String())
tmid, _ := toTrackingMessageID(mid)
pc.AckID(tmid)
}

In this case. We just want to convert MessageID into trackingMessageID. We do not care what it really is because it's not possible an invalid MessageID implementation.

So, original toTrackingMessageID() needs to require a careful look to use it correctly. I think it would be better to split it into two different method. toTrackingMessageID() just do the struct conversion, which it's more clearly. And when the new messageID type is added, we can just modify the checkMessageIDType.

Modifications

  • Refactor the toTrackingMessageID()
  • Add the checkMessageIDType() to check whether MessageID is user-defined.

Verifying this change

  • Make sure that the change passes the CI checks.

Documentation

  • Does this pull request introduce a new feature? no

@Gleiphir2769 Gleiphir2769 changed the title [Refactor]: Refactor the toTrackingMessageID() [refactor]: Refactor the toTrackingMessageID() Feb 28, 2023
@Gleiphir2769
Copy link
Contributor Author

Hi @BewareMyPower @RobertIndie @nodece @shibd, could you give a look when you are free?

@Gleiphir2769
Copy link
Contributor Author

/pulsarbot run-failure-checks

@Gleiphir2769
Copy link
Contributor Author

/pulsarbot run-failure-checks

Still doesn't work...

@BewareMyPower
Copy link
Contributor

The flaky test is related to #971

@Gleiphir2769
Copy link
Contributor Author

The flaky test is related to #971

@BewareMyPower yes, rerun CI failed again. Looks like CI is stucked in TestConsumerSeekByTimeOnPartitionedTopic now.

@BewareMyPower
Copy link
Contributor

I will review it tomorrow

if c.options.EnableDefaultNackBackoffPolicy || c.options.NackBackoffPolicy != nil {
mid := c.messageID(msg.ID())
if mid == nil {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even if we check the type of msgID, we can't remove this check, because there is other logic that returns nil

if partition < 0 || partition >= len(c.consumers) {
c.log.Warnf("invalid partition index %d expected a partition between [0-%d]",
partition, len(c.consumers))
return nil
}

We can add checkMessageIdType to c. messageID method, and keep this check.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great suggestion! I will fix it.

@RobertIndie RobertIndie merged commit d98c4f1 into apache:master Mar 7, 2023
@RobertIndie RobertIndie added this to the v0.10.0 milestone Mar 27, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants