Skip to content

[improve][broker] Recover batch deleted Index when cursor recover from metadataStore.#20018

Open
lifepuzzlefun wants to merge 4 commits intoapache:masterfrom
lifepuzzlefun:fix_cursor_recover_ignored_batch_ack_index
Open

[improve][broker] Recover batch deleted Index when cursor recover from metadataStore.#20018
lifepuzzlefun wants to merge 4 commits intoapache:masterfrom
lifepuzzlefun:fix_cursor_recover_ignored_batch_ack_index

Conversation

@lifepuzzlefun
Copy link
Contributor

@lifepuzzlefun lifepuzzlefun commented Apr 5, 2023

Motivation

fix #20585

just add the logic the same as recoverFromLedger(ManagedCursorInfo,VoidCallback) to the recover(VoidCallback)

PositionImpl position = new PositionImpl(positionInfo);
if (positionInfo.getIndividualDeletedMessagesCount() > 0) {
recoverIndividualDeletedMessages(positionInfo.getIndividualDeletedMessagesList());
}
if (config.isDeletionAtBatchIndexLevelEnabled()
&& positionInfo.getBatchedEntryDeletionIndexInfoCount() > 0) {
recoverBatchDeletedIndexes(positionInfo.getBatchedEntryDeletionIndexInfoList());
}
recoveredCursor(position, recoveredProperties, cursorProperties, lh);
callback.operationComplete();

i think both the logic is the same either recover from metadataStore or cursorLedger.

Modifications

add logic on recoverBatchDeletedIndex .

Verifying this change

  • Make sure that the change passes the CI checks.

(Please pick either of the following options)

This change is a trivial rework / code cleanup without any test coverage.

(or)

This change is already covered by existing tests, such as (please describe tests).

(or)

This change added tests and can be verified as follows:

(example:)

  • Added integration tests for end-to-end deployment with large payloads (10MB)
  • Extended integration test for recovery after broker failure

Does this pull request potentially affect one of the following parts:

If the box was checked, please highlight the changes

  • Dependencies (add or upgrade a dependency)
  • The public API
  • The schema
  • The default values of configurations
  • The threading model
  • The binary protocol
  • The REST endpoints
  • The admin CLI options
  • The metrics
  • Anything that affects deployment

Documentation

  • doc
  • doc-required
  • doc-not-needed
  • doc-complete

Matching PR in forked repository

PR in forked repository:

@github-actions github-actions bot added the doc-not-needed Your PR changes do not impact docs label Apr 5, 2023
@lhotari
Copy link
Member

lhotari commented Apr 6, 2023

Good catch. Please add tests to cover this PR. The test should fail without the fix and pass with the fix in this PR.

@lifepuzzlefun
Copy link
Contributor Author

@lhotari @codelipenghui @Technoboy- can you take a look ?

@github-actions
Copy link

github-actions bot commented Jun 9, 2023

The pr had no activity for 30 days, mark with Stale label.

@Technoboy- Technoboy- added this to the 3.2.0 milestone Jul 31, 2023
@Technoboy- Technoboy- modified the milestones: 3.2.0, 3.3.0 Dec 22, 2023
@coderzc coderzc modified the milestones: 3.3.0, 3.4.0 May 8, 2024
@lhotari lhotari requested a review from dlg99 October 9, 2024 14:36
@lhotari lhotari closed this Oct 10, 2024
@lhotari lhotari reopened this Oct 10, 2024
@lhotari lhotari modified the milestones: 4.0.0, 4.1.0 Oct 14, 2024
@lhotari lhotari removed this from the 4.1.0 milestone Jan 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

doc-not-needed Your PR changes do not impact docs ready-to-test Stale

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] Batch deleted Index lost when cursor recover from metadataStore

5 participants