From 361156e74cb29786a88d796982333efd1b214231 Mon Sep 17 00:00:00 2001 From: Yong Zhang Date: Wed, 15 May 2024 12:44:18 +0800 Subject: [PATCH] [improve][pip] PIP-350: Allow to disable the managedLedgerOffloadDeletionLagInMillis (#22688) --- pip/pip-350.md | 36 ++++++++++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+) create mode 100644 pip/pip-350.md diff --git a/pip/pip-350.md b/pip/pip-350.md new file mode 100644 index 0000000000000..f48771e7ee17d --- /dev/null +++ b/pip/pip-350.md @@ -0,0 +1,36 @@ +# PIP-350: Allow to disable the managedLedgerOffloadDeletionLagInMillis + +# Background knowledge + +https://pulsar.apache.org/docs/3.2.x/tiered-storage-overview/ +Pulsar provides the ability to offload the data from bookkeeper to the cloud storage with the tiered storage. +Once the data is offloaded to the cloud storage, the data in the bookkeeper can be deleted after a certain period of time. +We use the managedLedgerOffloadDeletionLagInMillis to control the deletion lag time for the offloaded data. +The default value of managedLedgerOffloadDeletionLagInMillis is 4 hours. It means the offloaded data will be deleted after 4 hours by default. + +# Motivation + +In some test scenarios, we want to disable the deletionLag and never delete the data from the bookkeeper. +Then when the tiered storage data is broken, we can still read the data from the bookkeeper. + +# Goals + +## In Scope + +Never deletes the bookkeeper data when the managedLedgerOffloadDeletionLagInMillis is set to -1. + +# Detailed Design + +## Design & Implementation Details + +Only need to check the value of managedLedgerOffloadDeletionLagInMillis in the ManagedLedgerImpl when it is going to delete the bookkeeper data. +https://github.com/apache/pulsar/blob/774a5d42e8342ee50395cf3626b9e7af27da849e/managed-ledger/src/main/java/org/apache/bookkeeper/mledger/impl/ManagedLedgerImpl.java#L2579 + +# Backward & Forward Compatibility + +Fully compatible. + +# Links + +* Mailing List discussion thread: https://lists.apache.org/thread/7tlpkcm2933ddg95kgrb42943r4gq3v9 +* Mailing List voting thread: https://lists.apache.org/thread/c3rh530dlwo6nhrdflpw0mjck85hhfbx