Support taking de-duplication snapshots based on time #8237
Comments
Jennifer88huang-zz
added
triage/week-42
type/feature
|
I will solve it |
This was referenced Nov 6, 2020
codelipenghui
pushed a commit
that referenced
this issue
Nov 9, 2020
Master Issue: #8237 ### Motivation Currently we take de-duplication snapshots based on size. If the topic has relatively low traffic, the de-duplication cursor will not move. This can cause messages that are not able to be deleted based on the retention policy. We should add a policy to take de-duplication snapshots based on time. ### Modifications add a monitor to take snapshot base on time ### Verifying this change TopicDuplicationTest#testDuplicationSnapshot
codelipenghui
pushed a commit
that referenced
this issue
Nov 11, 2020
Master Issue: #8237 ### Motivation Currently we take de-duplication snapshots based on size. If the topic has relatively low traffic, the de-duplication cursor will not move. This can cause messages that are not able to be deleted based on the retention policy. We should add a policy to take de-duplication snapshots based on time. ### Modifications add api to get/set/remove namespace-level policies ### Verifying this change TopicDuplicationTest.java
sijie
pushed a commit
that referenced
this issue
Nov 16, 2020
Master Issue: #8237 ### Motivation Currently we take de-duplication snapshots based on size. If the topic has relatively low traffic, the de-duplication cursor will not move. This can cause messages that are not able to be deleted based on the retention policy. We should add a policy to take de-duplication snapshots based on time. ### Modifications add api get/set/remove topic-level `DeduplicationSnapshotInterval` ### Verifying this change TopicDuplicationTest.java
flowchartsman
pushed a commit
to flowchartsman/pulsar
that referenced
this issue
Nov 17, 2020
Master Issue: apache#8237 ### Motivation Currently we take de-duplication snapshots based on size. If the topic has relatively low traffic, the de-duplication cursor will not move. This can cause messages that are not able to be deleted based on the retention policy. We should add a policy to take de-duplication snapshots based on time. ### Modifications add a monitor to take snapshot base on time ### Verifying this change TopicDuplicationTest#testDuplicationSnapshot
flowchartsman
pushed a commit
to flowchartsman/pulsar
that referenced
this issue
Nov 17, 2020
Master Issue: apache#8237 ### Motivation Currently we take de-duplication snapshots based on size. If the topic has relatively low traffic, the de-duplication cursor will not move. This can cause messages that are not able to be deleted based on the retention policy. We should add a policy to take de-duplication snapshots based on time. ### Modifications add api to get/set/remove namespace-level policies ### Verifying this change TopicDuplicationTest.java
flowchartsman
pushed a commit
to flowchartsman/pulsar
that referenced
this issue
Nov 17, 2020
Master Issue: apache#8237 ### Motivation Currently we take de-duplication snapshots based on size. If the topic has relatively low traffic, the de-duplication cursor will not move. This can cause messages that are not able to be deleted based on the retention policy. We should add a policy to take de-duplication snapshots based on time. ### Modifications add api get/set/remove topic-level `DeduplicationSnapshotInterval` ### Verifying this change TopicDuplicationTest.java
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Currently we take de-duplication snapshots based on size. If the topic has relatively low traffic, the de-duplication cursor will not move. This can cause messages that are not able to be deleted based on the retention policy. We should add a policy to take de-duplication snapshots based on time.
The text was updated successfully, but these errors were encountered: