How to Determine the Start and Completion of Kafka Scaling and Upgrades from CR Status?

[fengyinqiao]

Hello Strimzi Community,

I am currently working with the Strimzi Kafka Operator to manage Kafka clusters on Kubernetes. I have operations such as scaling up/down and upgrading the Kafka version that I need to perform and monitor efficiently.

I would like to know how I can use the Custom Resources (CRs) provided by Strimzi to track if these operations have started and when they are completed. Specifically, I am looking for ways to determine:

1. How to confirm that a scaling operation or version upgrade has been initiated?
2. How to check the completion of these operations through the CR status?

From what I understand, the status field in the Kafka CR should reflect the current state of the cluster, but I am unclear about which specific fields or indicators to look at that will reliably inform me about the start and end of these operations.

Any guidance on interpreting the CR status for operational events or if there are recommended practices for such monitoring in Strimzi would be greatly appreciated.

Thank you in advance for your help and suggestions!

Environment:

• Strimzi version:
• Kubernetes cluster: (e.g., Minikube, GKE)
• Infrastructure: (e.g., on-premises, cloud provider)

[scholzj]

How to confirm that a scaling operation or version upgrade has been initiated?

What exactly do you mean by that? You initiate the scaling operation. And you can watch when it completes through the status resources -> observed generations, ready pods etc.

How to check the completion of these operations through the CR status?

The .status section should provide you the information needed. Again, the different readiness flags, observed generation, version fields etc.

[ppatierno]

My use case, I want to know when a scale up or down is really finished. Even taking into account a restart of the operator.
I have the feeling that having something in the status explicitly related to scaling would be useful.

[fengyinqiao]

Oh yes agree. I was focusing on the scaling one.

@scholzj So, assume there is RollingUpdate status in the status.conditions filed(Currently only Ready is display), indicating that kafka is undergoing a rolling upgrade(either scaling or update), will it be better to user?

[scholzj]

Scaling has nothing to do with rolling update. There is also o condition RollingUpdate and just the fact that scaling or rolling update is going on does not mean that the cluster is not ready.

[fengyinqiao]

Scaling has nothing to do with rolling update. There is also o condition RollingUpdate and just the fact that scaling or rolling update is going on does not mean that the cluster is not ready.

Maybe the status can include a new filed, such as phase(value like Running | Scaling | RollingUpdate), would it be better?

[scholzj]

I think you are missing the point of how an operator works inside. First of all, scaling is nothing special. It just starts one or more new pods. Second, for rolling, everything happens on a node by node basis where the operator decides if any rolling is needed or not. So what you are suggesting is basically impossible to implement. Also, the use-case is not really clear. For scaling, you trigger the scaling event, you know what you expect to happen and you can already easily monitor that. Same for most rolling updates which are triggered by some change you do.