[Docs]: No information on triggering rollouts of db clusters with operator upgrade (with enabled in-place update)

[maryam-ismayilova-m10]

Is there an existing issue already for your request/idea?

• I have searched for an existing issue, and could not find anything. I believe this is a new documentation enhancement to be evaluated.

What problem in the existing documentation this issue aims to solve?

I have tried to upgrade the operator from version 1.25.1 to 1.26 (using the helm chart version 0.24.0) and this triggered an update on all my database clusters. This all despite the fact that i have enabled ENABLE_INSTANCE_MANAGER_INPLACE_UPDATES set to true .

I see the mention of probe changes in the release note(which probably is the reason for the need of a restart), but personally i did not expect it to be applied to any exisiting clusters. And even if so, I expected to be notified about this feature "breaking" the expectation that an operator upgrade is not going to trigger a rollout when inplace update is used.
I believe your documentation on upgrade steps and the release notes are missing this crucial information.

And perhaps it would be better to have the user decide whether such an enhancement need to be applied to all clusters or not (just like one can do turn of a restart using inplace update of instance manager)

Describe what additions need to be done to the documentation

I would expect this to be mentioned in installation and upgrade documentation here , as well as inside the release notes here

Describe what pages need to change in the documentation, if any

https://cloudnative-pg.io/documentation/1.26/installation_upgrade/

Describe what pages need to be removed from the documentation, if any

No response

Additional context

Previous version of cnpg operator: 1.25.1 with helm chart version 0.23.2
cnpg clusters deployed with cnpg cluster helm chart version 0.3.0

Log in cnpg operator pods with mention of a rollout needed:
original and target PodSpec differ in containers: container postgres differs in startup-probe

Backport?

N/A

Code of Conduct

• I agree to follow this project's Code of Conduct

[dosubot]

Hi @maryam-ismayilova-m10! I'm Dosu and I’m helping the cloudnative-pg team.

You are correct that the current documentation describes how enabling ENABLE_INSTANCE_MANAGER_INPLACE_UPDATES changes the default behavior of rolling updates during operator upgrades, but it does not explicitly warn that certain operator changes—such as modifications to probes—can still trigger rollouts of existing clusters, even with in-place updates enabled. The documentation states that by default, CloudNativePG issues a rolling update of the cluster every time the operator is updated, and that this can be changed by enabling in-place updates, which avoids updating the init container image and thus the Pod definition does not reflect the new operator version. However, it does not clarify that changes to the PodSpec (like probe changes) will still cause a rollout, as Kubernetes will detect a difference between the original and target PodSpec and trigger an update regardless of the in-place update setting documentation.

I did not find any existing issues or discussions that directly address this documentation gap or the specific scenario you described. The release notes for 1.26.0 do mention probe changes, but do not explicitly warn that these changes will trigger rollouts for existing clusters, even with in-place updates enabled.

Your suggestion to clarify this in the installation/upgrade documentation and release notes is well-founded. It would help set expectations for users that certain operator changes (such as probes or other PodSpec-affecting modifications) will still result in rollouts, and that the in-place update feature only prevents rollouts for changes limited to the instance manager itself, not for broader PodSpec changes.

If you need more technical details about how this is implemented or want to understand the rollout triggers in the codebase, let me know.

To reply, just mention my name @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}   

[usiegl00]

We experienced a similar issue where we had both the ENABLE_INSTANCE_MANAGER_INPLACE_UPDATES env set to true and primaryUpdateStrategy set to supervised on all our clusters. However, once we upgraded the operator the replicas were restarted to the newer 1.26 version immediately and the primary was left at our previous 1.25 version waiting for user action. We checked /run/manager version to see if the primary had been in-place updated, but found the version remained at 1.25. The cnpg cli plugin has two options for completing the supervised update, but neither allow for performing the in-place manager update on its own.

Running the kubectl cnpg restart command on the primary does perform the in-place update, but the postmaster was also restarted, which we did not expect to experience. Finally, the cluster status remained waiting for user action after the restart and required a kubectl cnpg promote of the follower to return to healthy.

[l00ptr]

We've observed the same behavior: our clusters were restarted after upgrading from CNPG 1.25.2 to 1.26.0, despite having ENABLE_INSTANCE_MANAGER_INPLACE_UPDATES set to true.

Upon inspection, the logs indicate that a Pod rollout was triggered due to a change in the startup probe configuration.

Specifically, we noticed the following difference in the probes before and after the upgrade:

Before upgrade:

Liveness:   http-get https://:8000/healthz delay=0s timeout=5s period=10s #success=1 #failure=3  
Readiness:  http-get https://:8000/readyz  delay=0s timeout=5s period=10s #success=1 #failure=3  
Startup:    http-get https://:8000/healthz delay=0s timeout=5s period=10s #success=1 #failure=360

After upgrade:

Liveness:   http-get https://:8000/healthz delay=0s timeout=5s period=10s #success=1 #failure=3  
Readiness:  http-get https://:8000/readyz  delay=0s timeout=5s period=10s #success=1 #failure=3  
Startup:    http-get https://:8000/startupz delay=0s timeout=5s period=10s #success=1 #failure=360

This change in the startup probe endpoint from /healthz to /startupz appears to be the cause of the rollout. Since we seem to know the origin of that problem, i am gonna make a merge request to say a few word about that problem into the upgrade documentation.