Prevent Accidental Kafka Topic Deletion in Strimzi Deployments

[spike83]

I hope everyone is doing well! I'm starting this thread to address a critical issue that many of us might have faced or fear facing in the future: accidental deletion of Kafka topics in our Strimzi deployments. This problem can occur due to various reasons, such as misconfigurations, missing resources in Kustomize, or other unexpected scenarios.

One effective measure to prevent accidental topic deletions is to disable the topic deletion feature directly on the Kafka brokers. This approach acts as a safeguard, ensuring that topics cannot be deleted even if a misconfiguration or an erroneous operation occurs. This can be achieved by setting delete.topic.enable=false in the broker configuration.

The downside of disabling the topic deletion is that you need to follow a certain procedure if you really want to delete a topic. To
do so you would need to enable it by doing a first rolling restart of the cluster, then you need to delete the topics manually and if you want to put it back you need to configure delete.topic.enable=false again and do another rolling restart.

I was wondering if we could prevent it in another way?

Only delete the topic if a certain annotation is set on the KafkaTopic resource:

An ideal way for my scenario would be one which still enables users to delete topics but preventing accidental deletion. If we could make the Topic Operator only delete if the user indicated so by e.g. setting an annotation.

Maybe also an extra finalizer could do it? This should prevent kubernetes from deleting the KafkaTopic resource in the first place. But as far as i tested it the Topic Operator will delete the topic if an extra finalizer is set.

Configure the Topic Operator with a less privileged user:

When we configure the Topic Operator with a less privileged user that is not allowed to delete topics we could achieve this as well.

The Topic Operator is by default using a SuperUser which will bypass the ACLs. This means we need to create another user for TopicOperator in order to limit the permissions.

Then another user must delete the topic by issuing some command to the kafka admin api (with another user extra for this purpose).

I tried this scenario. This works so far but the configuration of template > topicOperatorContainer seems to be a bit limited so that I had to create a dedicated deployment for Topic Operator.

What are your thoughts on this?

Thank you.

[scholzj]

Please keep in mind that using the Topic Operator is optional -> you can simply disable it and manage your topics in whatever way you want and whatever way suits your needs. It is not clear what version of Strimzi you use, but I think the past (the bidirectional topic operator) taught us that diverting from the standard operator pattern becomes a constant source of issues and problems. So at least I personally don't think we should start new avenues nd create new things on how to delete things. If you are afraid someone/something will delete the resources by mistake, you can possibly use the Kubernetes RBAC to prevent it?

[spike83]

I agree, Topic Operator is optional and maybe Kubernetes RBAC could be another option.

[spike83]

Thanks for the quick reply. I'm using the latest version, yes. And also the Unidirectional Topic Operator.
For me the operator pattern is perfectly fine but there i had the issue that I was unable to configure template > topicOperatorContainer with another user.

Problems I saw there:

• STRIMZI_SECURITY_PROTOCOL cannot be set
• valueFrom.. Secret is not possible
• changing the mount for another user key is not possible

Is the use of another user (especially not a superuser) a thing you would consider supporting?

[scholzj]

Thanks for the quick reply. I'm using the latest version, yes. And also the Unidirectional Topic Operator.
For me the operator pattern is perfectly fine but there i had the issue that I was unable to configure template > topicOperatorContainer with another user.

That is intentional. That would require to also create such a user. Such users would need to have the privileges, would need to exist, etc. With the Unidirectional Topic Operator - as it doesn't need ZooKeeper access anymore - if you want to use it with your own user, you should probably just deploy the standalone variant.