Updating from 0.18.18 -> 0.18.19 results in preview showing k8s resources getting replaced

[kennyjwilli]

Problem description

Updating my @pulumi/eks from 0.18.18 to 0.18.19 results in the pulumi up preview indicating that many/all of my k8s resources should be replaced.

My starting pulumi deps:

"@pulumi/pulumi": "1.9.1",
"@pulumi/kubernetes": "1.4.5",
"@pulumi/eks": "0.18.18",
"@pulumi/awsx": "0.19.1",
"@pulumi/aws": "1.18.0",

With these deps, I run pulumi up and receive the following output.

Previewing update (ComputeSoftware/dev):
     Type                 Name             Plan     Info
     pulumi:pulumi:Stack  cs-platform-dev           1 message
 
Diagnostics:
  pulumi:pulumi:Stack (cs-platform-dev):
    (node:26996) ExperimentalWarning: queueMicrotask() is experimental.

I then update to @pulumi/eks 0.18.19 and npm install. Running pulumi up results in many/all k8s resources getting replaced, according to the preview. Here are the first few lines from the preview:

     pulumi:pulumi:Stack                                               cs-platform-dev                                                   1 message
     ├─ eks:index:Cluster                                              k8s-cluster-dev                                                   
 ~   │  ├─ aws:eks:Cluster                                             k8s-cluster-dev-eksCluster                            update      [diff: ~tags]
 +-  │  ├─ pulumi:providers:kubernetes                                 k8s-cluster-dev-provider                              replace     [diff: ~kubeconfig]
 +-  │  ├─ pulumi:providers:kubernetes                                 k8s-cluster-dev-eks-k8s                               replace     [diff: ~kubeconfig]

The diff on the kubeconfig is not useful since the output side simply says => output<string>.

From the Pulumi Slack, @lukehoban said:

Theres a good change that this is just the preview being conservative - and that nothing will actually get replaced. But it's certainly dangerous.

[lukehoban]

In 0.18.19 the EKS package changed the default tags that are applied (added Name).

This results in an update on the aws:eks:Cluster. That leads to the outputs on the cluster being unknown (conservatively). That leads to the kubeconfig being unknown. That leads to the pulumi:providers:kubernetes provider needing to replace (conservative).

In practice, the outputs on the aws:eks:Cluster will not change, and should be treated as known during an update. This is the root issue. Due to this issue, effectively any change to the cluster (like adding tags) will result in preview proposing replacing any providers based on the kubeconfig.

But the change we made in defaults as part of the 0.18.18 to 0.18.19 upgrade is also a little unfortunate, as it leads to users hitting this without having made any changes at all to their code.

I believe users can technically work around this upgrade change by adding code like this to their cluster:

clusterTags: { Name: <any>undefined }

[metral]

For the pulumi-kubernetes provider, if the kubeconfig changes in a first-class Provider resource, the preview will show a replace for resources using that Provider.

From #362 (comment)

[metral]

Closing as https://github.com/pulumi/pulumi-eks/issues/352 is tracking the conservative replacement issue