Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix][broker] Fix issue with GetMessageIdByTimestamp can't find match messageId from compacted ledger #21600

Merged
merged 5 commits into from
Jan 20, 2024

Conversation

coderzc
Copy link
Member

@coderzc coderzc commented Nov 21, 2023

Motivation

Currently, if topic compaction is enabled and the original ledger is trimmed, we won't be able to get the correct messageID by publishTime, we should first try to find messageID from the compacted ledger. Since we introduced the findEntryByPublishTime method in PIP-286, we can easily implement it.

Modifications

  • Try to find messageID from the compacted ledger when getting messageID by publishTime.
  • Move findFirstMatchEntry and findFirstMatchIndexLoop methods to CompactedTopicImpl to cherry-pick this change into branches before 3.2.

Verifying this change

  • Make sure that the change passes the CI checks.

(Please pick either of the following options)

This change is a trivial rework / code cleanup without any test coverage.

(or)

This change is already covered by existing tests, such as (please describe tests).

(or)

This change added tests and can be verified as follows:

(example:)

  • Added integration tests for end-to-end deployment with large payloads (10MB)
  • Extended integration test for recovery after broker failure

Documentation

  • doc
  • doc-required
  • doc-not-needed
  • doc-complete

Matching PR in forked repository

PR in forked repository:

@github-actions github-actions bot added the doc-not-needed Your PR changes do not impact docs label Nov 21, 2023
@coderzc coderzc added this to the 3.2.0 milestone Nov 21, 2023
@coderzc coderzc self-assigned this Nov 22, 2023
@Technoboy- Technoboy- modified the milestones: 3.2.0, 3.3.0 Dec 22, 2023
@coderzc coderzc force-pushed the fix_find_timestamp_compaction branch from b9b2db3 to 66cd16e Compare January 12, 2024 04:50
@coderzc coderzc added type/bug The PR fixed a bug or issue reported a bug area/broker labels Jan 12, 2024
@coderzc coderzc merged commit 5cb12a5 into apache:master Jan 20, 2024
47 checks passed
@coderzc coderzc deleted the fix_find_timestamp_compaction branch January 20, 2024 01:37
@Technoboy- Technoboy- modified the milestones: 3.3.0, 3.2.0 Jan 22, 2024
Technoboy- pushed a commit that referenced this pull request Jan 22, 2024
Technoboy- pushed a commit that referenced this pull request Jan 31, 2024
coderzc added a commit that referenced this pull request Jan 31, 2024
…nd match messageId from compacted ledger (#21600)"

This reverts commit b5e4c36.
coderzc added a commit that referenced this pull request Jan 31, 2024
mukesh-ctds pushed a commit to datastax/pulsar that referenced this pull request Mar 1, 2024
… messageId from compacted ledger (apache#21600)

(cherry picked from commit b5e4c36)
mukesh-ctds pushed a commit to datastax/pulsar that referenced this pull request Mar 1, 2024
…nd match messageId from compacted ledger (apache#21600)"

This reverts commit b5e4c36.

(cherry picked from commit e53c19a)
mukesh-ctds pushed a commit to datastax/pulsar that referenced this pull request Mar 1, 2024
… messageId from compacted ledger (apache#21600)

(cherry picked from commit 5bb2264)
mukesh-ctds pushed a commit to datastax/pulsar that referenced this pull request Mar 6, 2024
… messageId from compacted ledger (apache#21600)

(cherry picked from commit b5e4c36)
mukesh-ctds pushed a commit to datastax/pulsar that referenced this pull request Mar 6, 2024
…nd match messageId from compacted ledger (apache#21600)"

This reverts commit b5e4c36.

(cherry picked from commit e53c19a)
mukesh-ctds pushed a commit to datastax/pulsar that referenced this pull request Mar 6, 2024
… messageId from compacted ledger (apache#21600)

(cherry picked from commit 5bb2264)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants