Enable/disable allowAutoTopicCreation per namespace #5395
Comments
sleungtoast
added
the
type/feature
|
@sleungtoast You maybe try admin.brokers().updateDynamicConfiguration(“allowAutoTopicCreation”, false); |
|
@huangdx0726 Hmm that could work, but if my app had admin privileges already, I could just have it create topics as an admin before using them, and I wouldn't have to temporarily turn on auto topic creation just so this single app can do so. |
|
@sleungtoast I am in a similar situation as you I would love to control this behavior at namespace level. For most of my clients I want allowAutoTopicCreation = false so I can funnel topic creation through a workflow and have an admin review the request etc.. However I have one client that uses only non-persistent topics and needs allowAutoTopicCreation = true. The way I have accomplished this is using namespace isolation policies to force that one client's namespaces to different brokers with different configurations. Not the best solution IMO but it will allow you to accommodate both use cases in one cluster. |
|
We would like to implement this. It looks like it can be done with configs similar to how deduplication can be enabled on a namespace basis: https://pulsar.apache.org/docs/en/pulsar-admin/#namespace-set-deduplication. Would there be any objections to doing this? |
…de (#6471) Fixes #5395 ### Motivation This change introduces a new namespace policy `autoTopicCreationOverride`, which will enable an override of broker `autoTopicCreation` settings on the namespace level. You may keep `autoTopicCreation` disabled for the broker and allow it on a specific namespace using this feature. ### Modifications - Add new namespace policy: `autoTopicCreationOverride` and associated API / CLI interface for setting and removing. Defaults to non-partitioned type, but also allows partitioned topics. - Modifies BrokerService: when checking `autoTopicCreation` configuration, the broker first retrieves namespace policies from zookeeper. If the `autoTopicCreationOverride` policy exists for that namespace then it uses those settings. If not, falls back to broker configuration. - Slight refactor to move `TopicType` enum to pulsar-common and add `autoTopicCreationOverride` to pulsar-common.
…de (#6471) Fixes #5395 This change introduces a new namespace policy `autoTopicCreationOverride`, which will enable an override of broker `autoTopicCreation` settings on the namespace level. You may keep `autoTopicCreation` disabled for the broker and allow it on a specific namespace using this feature. - Add new namespace policy: `autoTopicCreationOverride` and associated API / CLI interface for setting and removing. Defaults to non-partitioned type, but also allows partitioned topics. - Modifies BrokerService: when checking `autoTopicCreation` configuration, the broker first retrieves namespace policies from zookeeper. If the `autoTopicCreationOverride` policy exists for that namespace then it uses those settings. If not, falls back to broker configuration. - Slight refactor to move `TopicType` enum to pulsar-common and add `autoTopicCreationOverride` to pulsar-common. (cherry picked from commit fdc3a9b)
…override (apache#6471) Fixes apache#5395 ### Motivation This change introduces a new namespace policy `autoTopicCreationOverride`, which will enable an override of broker `autoTopicCreation` settings on the namespace level. You may keep `autoTopicCreation` disabled for the broker and allow it on a specific namespace using this feature. ### Modifications - Add new namespace policy: `autoTopicCreationOverride` and associated API / CLI interface for setting and removing. Defaults to non-partitioned type, but also allows partitioned topics. - Modifies BrokerService: when checking `autoTopicCreation` configuration, the broker first retrieves namespace policies from zookeeper. If the `autoTopicCreationOverride` policy exists for that namespace then it uses those settings. If not, falls back to broker configuration. - Slight refactor to move `TopicType` enum to pulsar-common and add `autoTopicCreationOverride` to pulsar-common.
Is your feature request related to a problem? Please describe.
Currently,
allowAutoTopicCreationis set as a broker configuration. Generally in my cluster, I want to set that to false to lock down auto topic creation. However, there are still some use cases where topics change a lot and dynamic topic creation would be easier.Describe the solution you'd like
I want to be able to turn on/off
allowAutoTopicCreationin certain namespaces.Describe alternatives you've considered
The text was updated successfully, but these errors were encountered: