Skip to content

Commit

Permalink
[improve][pip] PIP-350: Allow to disable the managedLedgerOffloadDele…
Browse files Browse the repository at this point in the history
…tionLagInMillis (#22688)
  • Loading branch information
zymap committed May 15, 2024
1 parent 3d26079 commit 361156e
Showing 1 changed file with 36 additions and 0 deletions.
36 changes: 36 additions & 0 deletions pip/pip-350.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
# PIP-350: Allow to disable the managedLedgerOffloadDeletionLagInMillis

# Background knowledge

https://pulsar.apache.org/docs/3.2.x/tiered-storage-overview/
Pulsar provides the ability to offload the data from bookkeeper to the cloud storage with the tiered storage.
Once the data is offloaded to the cloud storage, the data in the bookkeeper can be deleted after a certain period of time.
We use the managedLedgerOffloadDeletionLagInMillis to control the deletion lag time for the offloaded data.
The default value of managedLedgerOffloadDeletionLagInMillis is 4 hours. It means the offloaded data will be deleted after 4 hours by default.

# Motivation

In some test scenarios, we want to disable the deletionLag and never delete the data from the bookkeeper.
Then when the tiered storage data is broken, we can still read the data from the bookkeeper.

# Goals

## In Scope

Never deletes the bookkeeper data when the managedLedgerOffloadDeletionLagInMillis is set to -1.

# Detailed Design

## Design & Implementation Details

Only need to check the value of managedLedgerOffloadDeletionLagInMillis in the ManagedLedgerImpl when it is going to delete the bookkeeper data.
https://github.com/apache/pulsar/blob/774a5d42e8342ee50395cf3626b9e7af27da849e/managed-ledger/src/main/java/org/apache/bookkeeper/mledger/impl/ManagedLedgerImpl.java#L2579

# Backward & Forward Compatibility

Fully compatible.

# Links

* Mailing List discussion thread: https://lists.apache.org/thread/7tlpkcm2933ddg95kgrb42943r4gq3v9
* Mailing List voting thread: https://lists.apache.org/thread/c3rh530dlwo6nhrdflpw0mjck85hhfbx

0 comments on commit 361156e

Please sign in to comment.