Getting the stats of a non-persistent topic that has been cleaned causes it to re-appear #9029
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ses it to re-appear
|
/pulsarbot run-failure-checks |
rdhabalia
approved these changes
Dec 22, 2020
|
/pulsarbot run-failure-checks |
codelipenghui
approved these changes
Dec 23, 2020
|
/pulsarbot run-failure-checks |
1 similar comment
|
/pulsarbot run-failure-checks |
sijie
approved these changes
Dec 23, 2020
|
/pulsarbot run-failure-checks |
1 similar comment
|
/pulsarbot run-failure-checks |
|
This patch is good to go |
zzzming
pushed a commit
to datastax/pulsar
that referenced
this pull request
Jan 3, 2021
…ses it to re-appear (apache#9029) If a non-persistent topic is unused it is automatically deleted by Pulsar. If you then get the stats on that topic name using the REST API, it causes that topic to re-appear. For example, a non-persistent topic `public/bob/np` exists in a namespace. It is returned when using the `admin/v2/non-persistent/public/bob` endpoint: ``` ["non-persistent://public/bob/np"] ``` Since this topic is unused, it gets cleaned and no longer is returned by the endpoint: ``` [] ``` However, if you request the stats for that topic using the CLI (which calls the REST API), like this, you actually get a response (not a 404): ``` bin/pulsar-admin topics stats non-persistent://public/bob/np Warning: Nashorn engine is planned to be removed from a future JDK release { "msgRateIn" : 0.0, "msgThroughputIn" : 0.0, "msgRateOut" : 0.0, "msgThroughputOut" : 0.0, "bytesInCounter" : 0, "msgInCounter" : 0, "bytesOutCounter" : 0, "msgOutCounter" : 0, "averageMsgSize" : 0.0, "msgChunkPublished" : false, "storageSize" : 0, "backlogSize" : 0, "publishers" : [ ], "subscriptions" : { }, "replication" : { } } ``` And now the topic re-appears on the topic-list endpoint: ``` ["non-persistent://public/bob/np"] ``` When loading a temporary topic with createIfMissing = false do not try to create it, simply return an empty value. Add test case. This change added tests and can be verified as in the bug description. Run: pulsar-admin topics create non-persistent://public/default/tmp wait for the topic to be deleted run pulsar-admin topics stats non-persistent://public/default/tmp
codelipenghui
added a commit
that referenced
this pull request
Jan 3, 2021
… without topic auto-creation (#9107) ### Motivation After #9029 merged, the non-persistent topic can be created when enabling the topic auto-creation, Otherwise, the client will get a `Topic does not exist` exception. This looks like a concurrent merge related issue, but I'm not able to find another PR related to this issue. This PR is fixing the test that wants to subscribe to a partitioned non-persistent topic but disabled the topic auto-creation. Currently, the fix is enabling the topic auto-creation for the test. For non-persistent topics, we don't persist any metadata for it in the metadata server, so for users who want to use the non-persistent topic, they must enable the topic auto-creation.
codelipenghui
pushed a commit
that referenced
this pull request
Jan 7, 2021
…ses it to re-appear (#9029) If a non-persistent topic is unused it is automatically deleted by Pulsar. If you then get the stats on that topic name using the REST API, it causes that topic to re-appear. For example, a non-persistent topic `public/bob/np` exists in a namespace. It is returned when using the `admin/v2/non-persistent/public/bob` endpoint: ``` ["non-persistent://public/bob/np"] ``` Since this topic is unused, it gets cleaned and no longer is returned by the endpoint: ``` [] ``` However, if you request the stats for that topic using the CLI (which calls the REST API), like this, you actually get a response (not a 404): ``` bin/pulsar-admin topics stats non-persistent://public/bob/np Warning: Nashorn engine is planned to be removed from a future JDK release { "msgRateIn" : 0.0, "msgThroughputIn" : 0.0, "msgRateOut" : 0.0, "msgThroughputOut" : 0.0, "bytesInCounter" : 0, "msgInCounter" : 0, "bytesOutCounter" : 0, "msgOutCounter" : 0, "averageMsgSize" : 0.0, "msgChunkPublished" : false, "storageSize" : 0, "backlogSize" : 0, "publishers" : [ ], "subscriptions" : { }, "replication" : { } } ``` And now the topic re-appears on the topic-list endpoint: ``` ["non-persistent://public/bob/np"] ``` ### Modifications When loading a temporary topic with createIfMissing = false do not try to create it, simply return an empty value. Add test case. This change added tests and can be verified as in the bug description. Run: pulsar-admin topics create non-persistent://public/default/tmp wait for the topic to be deleted run pulsar-admin topics stats non-persistent://public/default/tmp (cherry picked from commit b860c05)
codelipenghui
added a commit
that referenced
this pull request
Jan 7, 2021
… without topic auto-creation (#9107) ### Motivation After #9029 merged, the non-persistent topic can be created when enabling the topic auto-creation, Otherwise, the client will get a `Topic does not exist` exception. This looks like a concurrent merge related issue, but I'm not able to find another PR related to this issue. This PR is fixing the test that wants to subscribe to a partitioned non-persistent topic but disabled the topic auto-creation. Currently, the fix is enabling the topic auto-creation for the test. For non-persistent topics, we don't persist any metadata for it in the metadata server, so for users who want to use the non-persistent topic, they must enable the topic auto-creation. (cherry picked from commit 21a3a3b)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
If a non-persistent topic is unused it is automatically deleted by Pulsar. If you then get the stats on that topic name using the REST API, it causes that topic to re-appear.
For example, a non-persistent topic
public/bob/npexists in a namespace. It is returned when using theadmin/v2/non-persistent/public/bobendpoint:Since this topic is unused, it gets cleaned and no longer is returned by the endpoint:
However, if you request the stats for that topic using the CLI (which calls the REST API), like this, you actually get a response (not a 404):
And now the topic re-appears on the topic-list endpoint:
Modifications
When loading a temporary topic with createIfMissing = false do not try to create it, simply return an empty value.
Add test case.
This change added tests and can be verified as in the bug description.
Run:
pulsar-admin topics create non-persistent://public/default/tmp
wait for the topic to be deleted
run
pulsar-admin topics stats non-persistent://public/default/tmp