Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix][offload] Don't cleanup data when offload met MetaStore exception #21686

Merged
merged 2 commits into from
Dec 7, 2023

Conversation

zymap
Copy link
Member

@zymap zymap commented Dec 7, 2023

Motivation

#17915 changes the fix #17512 which lead the offload data is deleted when metadata store exception happened. Then the ledger can not be read.

The logs shows

Failed to update offloaded metadata for the ledgerId 6197907, the offloaded data will not be cleaned up

But the ledger deleted. Then managed ledger failed to open it

Error opening ledger for reading at position 6197907:0

Modifications

Verifying this change

  • Make sure that the change passes the CI checks.

(Please pick either of the following options)

This change is a trivial rework / code cleanup without any test coverage.

(or)

This change is already covered by existing tests, such as (please describe tests).

(or)

This change added tests and can be verified as follows:

(example:)

  • Added integration tests for end-to-end deployment with large payloads (10MB)
  • Extended integration test for recovery after broker failure

Does this pull request potentially affect one of the following parts:

If the box was checked, please highlight the changes

  • Dependencies (add or upgrade a dependency)
  • The public API
  • The schema
  • The default values of configurations
  • The threading model
  • The binary protocol
  • The REST endpoints
  • The admin CLI options
  • The metrics
  • Anything that affects deployment

Documentation

  • doc
  • doc-required
  • doc-not-needed
  • doc-complete

Matching PR in forked repository

PR in forked repository:

…xception

---

### Motivation

apache#17915 changes the fix apache#17512 which lead the offload data
is deleted when metadata store exception happened. Then the
ledger can not be read.

The logs shows
```
Failed to update offloaded metadata for the ledgerId 6197907, the offloaded data will not be cleaned up
```
But the ledger deleted. Then managed ledger failed to open it
```
Error opening ledger for reading at position 6197907:0
```
@zymap zymap added this to the 3.2.0 milestone Dec 7, 2023
@zymap zymap self-assigned this Dec 7, 2023
@github-actions github-actions bot added the doc-not-needed Your PR changes do not impact docs label Dec 7, 2023
@zymap zymap changed the title [fix][tiered-storage] Don't cleanup data when offload met MetaStore exception [fix][offload] Don't cleanup data when offload met MetaStore exception Dec 7, 2023
@zymap
Copy link
Member Author

zymap commented Dec 7, 2023

/pulsarbot rerun-failure-checks

@codecov-commenter
Copy link

Codecov Report

Merging #21686 (35bf96b) into master (ab77ca2) will decrease coverage by 0.06%.
Report is 1 commits behind head on master.
The diff coverage is 100.00%.

Additional details and impacted files

Impacted file tree graph

@@             Coverage Diff              @@
##             master   #21686      +/-   ##
============================================
- Coverage     73.39%   73.34%   -0.06%     
+ Complexity    32763    32738      -25     
============================================
  Files          1893     1893              
  Lines        140753   140754       +1     
  Branches      15503    15503              
============================================
- Hits         103306   103232      -74     
- Misses        29346    29415      +69     
- Partials       8101     8107       +6     
Flag Coverage Δ
inttests 24.16% <0.00%> (+0.04%) ⬆️
systests 24.64% <0.00%> (-0.19%) ⬇️
unittests 72.63% <100.00%> (-0.04%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Coverage Δ
...che/bookkeeper/mledger/impl/ManagedLedgerImpl.java 80.78% <100.00%> (+0.13%) ⬆️

... and 60 files with indirect coverage changes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

8 participants