Increase the rate of topic creation

[suiyuzeng]

Motivation

The count of topic created in 1 second is less than 200, when the topic count of the namespace is about 200,000..

Modifications

The reason is that the time cost of topic exist checking is too long when the topic count is large.
In org.apache.pulsar.broker.admin.AdminResource#checkTopicExistsAsync, all the topics would be checked one by one. As the count of topic grows, the time cost will become longer.
Can we use org.apache.pulsar.broker.namespace.NamespaceService#checkTopicExists to check if the topic is exits?

Verifying this change

• Make sure that the change passes the CI checks.

(Please pick either of the following options)

This change is a trivial rework / code cleanup without any test coverage.

(or)

This change is already covered by existing tests, such as (please describe tests).

(or)

This change added tests and can be verified as follows:

Does this pull request potentially affect one of the following parts:

If yes was chosen, please highlight the changes

• Dependencies (does it add or upgrade a dependency): (no)
• The public API: (no)
• The schema: (no)
• The default values of configurations: (no)
• The wire protocol: ( no)
• The rest endpoints: (no)
• The admin cli options: (no)
• Anything that affects deployment: ( no )

Documentation

Check the box below and label this PR (if you have committer privilege).

Need to update docs?

• doc-required

  (If you need help on updating docs, create a doc issue)

• no-need-doc

  code optimize

• doc

  (If this PR contains doc changes)

[codelipenghui]

Nice catch! But I think this will break the current behavior. Currently, we do not allow the duplicate topic name with the persistent or non-persistent domain, with this change it will only check the persistent topic duplication.

I think we should add the non-persistent topic duplication check back.

[suiyuzeng]

Nice catch! But I think this will break the current behavior. Currently, we do not allow the duplicate topic name with the persistent or non-persistent domain, with this change it will only check the persistent topic duplication.

I think we should add the non-persistent topic duplication check back.

Got it. I will update later.

[suiyuzeng]

@codelipenghui hi, can you please take a look?

[315157973]

We have multiLayerTopicsMap in BrokerService, we don't need to traverse all topics under the namespace, and then traverse again to determine whether topic exists.
We only need to traverse the bundles, and then directly use map.contains , usually the number of bundles is not too much

[eolivelli]

very nice catch !
I am happy to see an improvement here

do we have specific test cases around this function ?

[suiyuzeng]

@315157973 @eolivelli thanks，i will Optimize the code and add some test about this function.

[315157973]

@suiyuzeng Hello, please go on

[suiyuzeng]

@suiyuzeng Hello, please go on

Sorry, a little busy recently~
Find another problem, such as the test case "testCreatePartitionedTopicHavingNonPartitionTopicWithPartitionSuffix". Some non partition topic has the partition suffix. In this case, we can not use the cache to check anymore. There is some checking about this issue in the current code, to ensure that we can not create no no-parititon topic whit -partition-xxx suffix. How about add a config? If a cluster has the no-partition whit the suffix, use the current way to check duplication. If no no-paritition topic whit the suffix, use the cache to check.

[hangc0276]

@suiyuzeng Hello, please go on

Sorry, a little busy recently~
Find another problem, such as the test case "testCreatePartitionedTopicHavingNonPartitionTopicWithPartitionSuffix". Some non partition topic has the partition suffix. In this case, we can not use the cache to check anymore. There is some checking about this issue in the current code, to ensure that we can not create no no-parititon topic whit -partition-xxx suffix. How about add a config? If a cluster has the no-partition whit the suffix, use the current way to check duplication. If no no-paritition topic whit the suffix, use the cache to check.

@suiyuzeng Nice catch!

1. I am cutting 2.8.1, would you have time to address @315157973 's comments?
2. for the restriction for reserved key-words for topic name, such as -partition-xx suffix for non-partitioned topic name, we have started a discussion for this issue: https://lists.apache.org/list.html?dev@pulsar.apache.org

[hangc0276]

move to 2.8.2

[315157973]

move to 2.8.3

[github-actions]

@suiyuzeng:Thanks for your contribution. For this PR, do we need to update docs?
(The PR template contains info about doc, which helps others know more about the changes. Can you provide doc-related info in this and future PR descriptions? Thanks)

[michaeljmarshall]

Removing the release/2.8.3 label as this change will target master and probably won't be cherry picked back to stable branches because it is an optimization.

[tisonkun]

Closed as stale and conflict. Please rebase and resubmit the patch if it's still relevant.

I believe this patch is worth continuing, but holding a stale patch doesn't work. If @suiyuzeng can rebase and resubmit the patch, I'm glad to follow the PR lifecycle. Otherwise, I may pick it up later. Close.