docs: Warn on key rotations during upgrades

[ upstream commit b639eab ]

In general, it is not recommended to carry several admin. operations on
the cluster at the same time, as it can make troubleshooting in case of
issues a lot more complicated. Mixing operations is also less likely to
be covered in CI so more likely to hit corner cases.

Performing IPsec key rotations during Cilium up/downgrades is one such
case. Let's document it explicitly to discourage users from doing that.

Signed-off-by: Paul Chaignon <paul.chaignon@gmail.com>
Signed-off-by: Sebastian Wicki <sebastian@isovalent.com>

Documentation/security/network/encryption-ipsec.rst

@@ -162,6 +162,12 @@ commands:
 Key Rotation
 ============
 
+.. attention::
+
+   Key rotations should not be performed during upgrades and downgrades. That
+   is, all nodes in the cluster (or clustermesh) should be on the same Cilium
+   version before rotating keys.
+
 To replace cilium-ipsec-keys secret with a new key:
 
 .. code-block:: shell-session