You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When trying to get all the nodes to the same state after doing a canary upgrade, we run into a situation where the check for StatefulSet being completed will always fail and the operator will output:
2024-05-16T05:41:28.555Z INFO waiting for upgrade to finish on statefulset {"controller": "cassandradatacenter_controller", "controllerGroup": "cassandra.datastax.com", "controllerKind": "CassandraDatacenter", "CassandraDatacenter": {"name":"dc1","namespace":"test-canary-upgrade"}, "namespace": "test-canary-upgrade", "name": "dc1", "reconcileID": "56346c1d-4130-4020-b659-2f18f8ca4a0c", "namespace": "test-canary-upgrade", "datacenterName": "dc1", "clusterName": "cluster1", "statefulset": "cluster1-dc1-r1-sts", "replicas": 3, "readyReplicas": 3, "currentReplicas": 2, "updatedReplicas": 1}
There's no recovery from this, since what cass-operator expects is:
What happened?
When trying to get all the nodes to the same state after doing a canary upgrade, we run into a situation where the check for StatefulSet being completed will always fail and the operator will output:
There's no recovery from this, since what cass-operator expects is:
However, the StatefulSet itself will output the following status (in 1.28):
This is the final and correct state. Sadly, there's no way to tell from the status if the canary upgrade was actually reached correctly.
What did you expect to happen?
We should be able to continue the upgrade by simply removing the canary upgrade requirements.
How can we reproduce it (as minimally and precisely as possible)?
See the test in #654
cass-operator version
1.20
Kubernetes version
1.28
Method of installation
No response
Anything else we need to know?
No response
The text was updated successfully, but these errors were encountered: