Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Inactive non-persistent subscriptions not purged #5759

Closed
wmccarley opened this issue Nov 27, 2019 · 2 comments · Fixed by #8166
Closed

Inactive non-persistent subscriptions not purged #5759

wmccarley opened this issue Nov 27, 2019 · 2 comments · Fixed by #8166
Assignees
@315157973
Labels
help wanted release/2.6.2 type/bug The PR fixed a bug or issue reported a bug
Milestone
2.7.0

Comments

@wmccarley
Copy link
Contributor

Describe the bug

I have set up a cluster with brokers that only serve non-persistent topics. Furthermore, I have set the topic/subscription purge intervals/frequencies to low values to satisfy the use case I am focused on. However I am not seeing topics get cleaned up due to inactivity.

In the original pull request (1352) by @yush1ga that implemented the inactive subscription purge feature the checkInactiveSubscriptions method was not implemented for NonPersistent topics:
https://github.com/apache/pulsar/pull/1352/files#diff-48abd445bf0d38e83cc8966f765f6615

Isn't the side effect of this that unused subscriptions will never be cleared on non-persistent topics? In turn this prevents the broker from purging the topic as inactive if there are also no producers?

Is this correct?

My concern is that in an environment where I need lots of short-lived (read: ephemeral) non-persistent topics I will accumulate topic metadata as older topics are abandoned but never fully cleaned up...

Expected behavior

I would expect that the subscription inactivity and topic inactivity settings would apply to non-persistent topics as well.
@wmccarley wmccarley added the type/bug The PR fixed a bug or issue reported a bug label Nov 27, 2019
@jiazhai
Copy link
Member

jiazhai commented Nov 28, 2019

Thanks for the detailed reports @wmccarley . This is a feature that need to fix for non-persistent topics. This should be a direct fix that need following #1352 have done, mark this as help wanted.

@sijie sijie mentioned this issue Dec 27, 2019
Closed
@315157973 315157973 mentioned this issue Sep 30, 2020
Merged
@codelipenghui codelipenghui added this to the 2.7.0 milestone Oct 4, 2020
@315157973
Copy link
Contributor

I will work on it

sijie pushed a commit that referenced this issue Oct 5, 2020
@315157973
[ISSUE #5759] Clean inactive non-persistent subscriptions (#8166)
6711b71 
Fixes #5759

### Motivation
unused subscriptions will never be cleared on non-persistent topics

### Modifications
Add the `lastActive` attribute for non-persistent subscriptions. In the `checkInactiveSubscriptions` method of `NonPersistentTopic`, eligible subscriptions will be cleaned up

### Verifying this change
org.apache.pulsar.client.api.NonDurableSubscriptionTest#testDeleteInactiveNonPersistentSubscription
wolfstudy pushed a commit that referenced this issue Oct 30, 2020
@315157973 @wolfstudy
[ISSUE #5759] Clean inactive non-persistent subscriptions (#8166)
4d6f343 
Fixes #5759

### Motivation
unused subscriptions will never be cleared on non-persistent topics

### Modifications
Add the `lastActive` attribute for non-persistent subscriptions. In the `checkInactiveSubscriptions` method of `NonPersistentTopic`, eligible subscriptions will be cleaned up

### Verifying this change
org.apache.pulsar.client.api.NonDurableSubscriptionTest#testDeleteInactiveNonPersistentSubscription

(cherry picked from commit 6711b71)
huangdx0726 pushed a commit to huangdx0726/pulsar that referenced this issue Nov 13, 2020
@315157973 @huangdx0726
[ISSUE apache#5759] Clean inactive non-persistent subscriptions (apac…
cdbab5e 
…he#8166)

Fixes apache#5759

### Motivation
unused subscriptions will never be cleared on non-persistent topics

### Modifications
Add the `lastActive` attribute for non-persistent subscriptions. In the `checkInactiveSubscriptions` method of `NonPersistentTopic`, eligible subscriptions will be cleaned up

### Verifying this change
org.apache.pulsar.client.api.NonDurableSubscriptionTest#testDeleteInactiveNonPersistentSubscription
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@315157973 315157973

Labels
help wanted release/2.6.2 type/bug The PR fixed a bug or issue reported a bug
Projects
None yet
Milestone
2.7.0
Development

Successfully merging a pull request may close this issue.

[ISSUE #5759] Clean inactive non-persistent subscriptions 315157973/pulsar
4 participants
@jiazhai @315157973 @codelipenghui @wmccarley