From a4d737550938c28f784630505d5f862479632677 Mon Sep 17 00:00:00 2001 From: zymap Date: Fri, 10 May 2024 11:24:01 +0800 Subject: [PATCH 1/3] [improve][pip] PIP-350: Allow to disable the managedLedgerOffloadDeletionLagInMillis --- ### Motivation Allow to disable the managedLedgerOffloadDeletionLagInMillis --- pip/pip-350.md | 39 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 39 insertions(+) create mode 100644 pip/pip-350.md diff --git a/pip/pip-350.md b/pip/pip-350.md new file mode 100644 index 0000000000000..910048e52ca64 --- /dev/null +++ b/pip/pip-350.md @@ -0,0 +1,39 @@ +# PIP-350: Allow to disable the managedLedgerOffloadDeletionLagInMillis + +# Background knowledge + +https://pulsar.apache.org/docs/3.2.x/tiered-storage-overview/ +Pulsar provides the ability to offload the data from bookkeeper to the cloud storage with the tiered storage. +Once the data is offloaded to the cloud storage, the data in the bookkeeper can be deleted after a certain period of time. +We use the managedLedgerOffloadDeletionLagInMillis to control the deletion lag time for the offloaded data. +The default value of managedLedgerOffloadDeletionLagInMillis is 4 hours. It means the offloaded data will be deleted after 4 hours by default. + +# Motivation + +In some test scenarios, we want to disable the deletionLag and never delete the data from the bookkeeper. +Then when the tiered storage data is broken, we can still read the data from the bookkeeper. + +# Goals + +## In Scope + +Never deletes the bookkeeper data when the managedLedgerOffloadDeletionLagInMillis is set to -1. + +# Detailed Design + +## Design & Implementation Details + +Only need to check the value of managedLedgerOffloadDeletionLagInMillis in the ManagedLedgerImpl when it is going to delete the bookkeeper data. +https://github.com/apache/pulsar/blob/774a5d42e8342ee50395cf3626b9e7af27da849e/managed-ledger/src/main/java/org/apache/bookkeeper/mledger/impl/ManagedLedgerImpl.java#L2579 + +# Backward & Forward Compatibility + +Fully compatible. + +# Links + + +* Mailing List discussion thread: +* Mailing List voting thread: From 921e0295d40f09b9f5f34a673183033367739a7f Mon Sep 17 00:00:00 2001 From: zymap Date: Fri, 10 May 2024 11:33:13 +0800 Subject: [PATCH 2/3] add the discussion link --- pip/pip-350.md | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/pip/pip-350.md b/pip/pip-350.md index 910048e52ca64..492f95d03a3d9 100644 --- a/pip/pip-350.md +++ b/pip/pip-350.md @@ -32,8 +32,5 @@ Fully compatible. # Links - -* Mailing List discussion thread: +* Mailing List discussion thread: https://lists.apache.org/thread/7tlpkcm2933ddg95kgrb42943r4gq3v9 * Mailing List voting thread: From c5bc923e2cf51672a19cc9b2e40cadca5c3999b8 Mon Sep 17 00:00:00 2001 From: zymap Date: Wed, 15 May 2024 10:43:06 +0800 Subject: [PATCH 3/3] Update the voting thread --- pip/pip-350.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pip/pip-350.md b/pip/pip-350.md index 492f95d03a3d9..f48771e7ee17d 100644 --- a/pip/pip-350.md +++ b/pip/pip-350.md @@ -33,4 +33,4 @@ Fully compatible. # Links * Mailing List discussion thread: https://lists.apache.org/thread/7tlpkcm2933ddg95kgrb42943r4gq3v9 -* Mailing List voting thread: +* Mailing List voting thread: https://lists.apache.org/thread/c3rh530dlwo6nhrdflpw0mjck85hhfbx