[Bug] Saving unchanged cluster leads to update status and cluster changes

[rmweir]

Setup

• Rancher version:
  2.6.6 and 2.6.7 backend pointing at latest UI for dev.

Describe the bug

Saving unchanged cluster leads to update status and cluster changes.

To Reproduce

1. Create a cluster.
2. Wait for cluster to go to active.
3. Go to edit the cluster.
4. Switch to yaml (I think this is actually optional).
5. Hit save.

Result
Fields are added and the cluster goes into updating.

Expected Result

The cluster should remain unchanged and should not go into updating.

Additional context

<!--Add any other context about the problem here. -->
The diff between cluster before save and after save:
<  generation: 27
---
>  generation: 28
25c25
<  resourceVersion: "114966"
---
>  resourceVersion: "116054"
28c28,29
<  agentImageOverride: ""
---
>  agentEnvVars: []
>  agentImageOverride: null
30,32c31,36
<  description: ""
<  desiredAgentImage: ""
<  desiredAuthImage: ""
---
>  clusterTemplateRevisionName: null
>  defaultClusterRoleForProjectMembers: null
>  defaultPodSecurityPolicyTemplateName: null
>  description: null
>  desiredAgentImage: null
>  desiredAuthImage: null
50c54,57
<    nodelocal: {}
---
>    linearAutoscalerParams: {}
>    nodelocal:
>     updateStrategy: {}
>    updateStrategy: {}
98c105,113
<  scheduledClusterScan: {}
---
>  scheduledClusterScan:
>   enabled: false
>   scanConfig:
>    cisScanConfig:
>     overrideBenchmarkVersion: rke-cis-1.5
>     profile: permissive
>   scheduleConfig:
>    cronSchedule: 0 0 * * *
>    retention: 24

[cbron]

Potentially a blocker, adding label to discuss.

[catherineluse]

Just reproduced it. Also noticed the same thing happens if I save the cluster from the edit config form without switching to YAML.

[catherineluse]

@rmweir When I go to edit a cluster, click Edit as YAML, and click Show Diff, I see these changes to hostNamePrefix. This is for a 4-node cluster with two node pools:

And this is the diff for the initial save of a single-node cluster:

Should the UI wait to apply that change until after the user has changed something? Or is the problem that these values were not appropriately applied when the cluster was first provisioned?

After saving the cluster that way, if I save it again, it doesn't add more values or put the cluster in an updating state, because the values are only added if they don't already exist.

[catherineluse]

Gathered a little more information:

• The code that adds the node pool prefix was added by Vince in March 2021/Rancher v2.5.16 75ec34c#diff-7d87fdaaff449429c3eb17683445ad23033fb135e86941814b8f4aaa43b94326R301

if ( !entry.pool.hostnamePrefix ) {
          entry.pool.hostnamePrefix = `${ prefix }-`;
        }

• The code that adds the default values for the authorized cluster endpoint was added by Vince in May 2021/Rancher v2.6.5) 6de4934#diff-f32dce3947d86b7c96a1cc70b768e414984a5025d2410c2094a213bbf5ae9b2cR27

data() {
    if ( isEmpty(this.value?.spec?.localClusterAuthEndpoint) ) {
      set(this.value, 'spec.localClusterAuthEndpoint', {
        enabled: false,
        caCerts: '',
        fqdn:    '',
      });
    }

• The code that sets ignoreErrors to false was added by Vince in May 2020/Rancher v2.6.5 afdc49d#diff-a14eebe4bb7d477d4e4239e9004d2dd737fd44468f337129c232f54f4bfa732dR13

const DEFAULTS = {
  deleteEmptyDirData:              false, // Show; Kill pods using emptyDir volumes and lose the data
  disableEviction:                 false, // Hide; false = evict pods, true = delete pods
  enabled:                         false, // Show; true = Nodes must be drained before upgrade; false = YOLO
  force:                           false, // Show; true = Delete standalone pods, false = fail if there are any
  gracePeriod:                     -1, // Show; Pod shut down time, negative value uses pod default
  ignoreDaemonSets:                true, // Hide; true = work, false = never work because there's always daemonSets
  ignoreErrors:                    false, // Hide; profit?
  skipWaitForDeleteTimeoutSeconds: 0, // Hide; If the pod deletion time is older than this > 0, don't wait, for some reason
  timeout:                         120, // Show; Give up after this many seconds
};

[gaktive]

Taking blocker off but leaving as a priority/0. This has been around for a few versions so alas, it's not a true regression but it's still weird.

[rmweir]

@catherineluse after waiting a while the fields get over written and saving does the same thing again. I'm not sure, we would have to compare to what the behavior was before this started happening. I think the UI should not be changing anything about the cluster, if there is an empty/nil value it should be left as is.

[catherineluse]

@rmweir What's the Rancher version where you remember it working properly?

[rmweir]

@catherineluse I don't know which rancher version worked properly. I just know it's in 2.6.6+ at least.

[catherineluse]

Here's the behavior for editing an RKE2 cluster in 2.6.3:

and 2.6.0:

I couldn't reproduce it for RKE1.

Also, after saving the RKE2 cluster, it sometimes went into updating mode when saving from the form view, but not from the YAML view if the YAML diff showed no changes.

[gaktive]

@catherineluse indicated that backend couldn't repro this consistently though they do see some issue. They've pushed their corresponding issue to 2.7.1, so we'll do the same.

[mantis-toboggan-md]

Sure I can look @thaneunsoo; did you see this on RKE1, RKE2, and K3s?

[thaneunsoo]

@mantis-toboggan-md yes, for rke1,rke2,k3s

[mantis-toboggan-md]

Interesting: RKE1 provisioning is done through the old UI so there's either the same problem in both UIs or there is another problem w/ editing but not changing anything on the backend... I'm not seeing the behaviour you are on my own setup but it is older; I'm making a fresh one and looking into this further now

[mantis-toboggan-md]

@thaneunsoo I was able to repro this behavior with rke2 and k3s; a fix is now merged: #7218

What I saw specifically was that the rke2 or k3s cluster would go into an updating state after saving from a different view than the cluster had previously been saved in. So: provision a cluster without hitting 'view as yaml', go to 'edit config', then click 'view as yaml', save, the cluster updates despite you not changing anything. Again go to 'edit config', click 'view as yaml', save, the cluster does not change state. Go to 'edit config', save without clicking 'view as yaml', the cluster goes into updating...you get the idea

I'm having a harder time reproducing this consistently with RKE1. Could we potentially file a separate issue for that @thaneunsoo ?

[thaneunsoo]

Test Environment:

Rancher version: v2.7-head 9f1e043
Rancher cluster type: HA
Docker version: 20.10

Downstream cluster type: RKE2/K3s

---

Testing:

1. Provision RKE2/K3s cluster
2. Wait for cluster to become active
   a. Click Edit Config > Save
   b. Click Edit Config > View as YAML > Save

Results
Clusters do not go into Updating state. I did notice that the first I performed either of steps 2a or 2b, the cluster would go into In Progress for a split second and then go to active. If the user took their eyes off the screen for a second, they would miss this. This seems to be okay as there is editing in progress. Closing this issue as fixed

[snasovich]

@gaktive @nwmac , we will need this fixed in 2.6.10 as well. I've tried to create backport, but it looks like backport/forwardport bot is not available in this repo.
Could you please create a backport to ensure this is fixed in 2.6.10?
cc: @sowmyav27