Overhaul logic for resource diffing (#2445)

This commit incorporates several significant changes to the resource diff logic. These changes are summarized below, followed by detailed explanations of each:

1. Drop usage of the "kubectl.kubernetes.io/last-applied-configuration" annotation.
2. Compute preview diffs using resource inputs rather than making a dry-run API call.
3. Automatically update `.metadata.managedFields` to work with resources that were managed with client-side apply, and later upgraded to use server-side apply.
4. Fix a bug with the diff calculation so that resource drift is detected accurately after a refresh.

---

1. Previous versions of the provider used the "kubectl.kubernetes.io/last-applied-configuration" annotation to store a copy of the configuration as part of the Kubernetes resource. This annotation was used for input diff computation in client-side apply mode. Specifically, input diffs were computed between the new inputs and the last-applied-configuration value from the previous live state. Using this annotation resulted in a large number of reported issues (30+) for the provider, and it will no longer be used.

    14916d3 added logic to prune a map based on a target map. This logic is used to prune live state to match the previous inputs before performing an input diff computation. This avoids the need to store the last-applied-configuration.

2. Server-side apply mode in 3.x versions of the provider performed several network-bound API calls as part of the resource diff computation. This led to poor performance and diverged from the way that other Pulumi providers compute resource diffs. This behavior was brought in line with other providers, and now previews resource changes using the client-side inputs. Accepted changes are still updated using server-side apply. Note that this now requires the stack to be refreshed to detect resource drift in both client-side apply and server-side apply modes.

3. In server-side apply (SSA) mode, Kubernetes uses information in the `.metadata.managedFields` property to determine ownership of each field in a resource. Kubernetes v1.18+ sets the managedFields property in both client-side and server-side apply modes. Operations performed using client-side mode set a field manager with an operation type of "Update", while server-side operations use "Apply". Kubernetes treats these as separate field managers, which can lead to problems during server-side updates. To avoid this class of problems, the provider automatically takes ownership of fields managed by known client-side field managers during the first SSA-enabled update. This change should be transparent to users. See kubernetes/kubernetes#99003 for additional details.

4. Previous versions of the provider contained a bug where previous inputs rather than live state were used to compute a diff. This prevented the provider from detecting resource drift caused by other controllers. This bug was fixed so that drift will be corrected on update after a refresh is performed.

---------

Co-authored-by: Ramon Quitales <ramon@pulumi.com>

CHANGELOG.md

@@ -4,22 +4,22 @@
 
 Breaking changes:
 
-- Enable Server-side Apply by default (https://github.com/pulumi/pulumi-kubernetes/pull/2398)
 - Remove deprecated enableDryRun provider flag (https://github.com/pulumi/pulumi-kubernetes/pull/2400)
 - Remove deprecated helm/v2 SDK (https://github.com/pulumi/pulumi-kubernetes/pull/2396)
 - Remove deprecated enableReplaceCRD provider flag (https://github.com/pulumi/pulumi-kubernetes/pull/2402)
 - Drop support for Kubernetes clusters older than v1.13 (https://github.com/pulumi/pulumi-kubernetes/pull/2414)
 - Make all resource output properties required (https://github.com/pulumi/pulumi-kubernetes/pull/2422)
-- Drop support for legacy pulumi.com/initialApiVersion annotation (https://github.com/pulumi/pulumi-kubernetes/pull/2443)
 
 Other changes:
 
-- Automatically fall back to client-side preview if server-side preview fails (https://github.com/pulumi/pulumi-kubernetes/pull/2419)
 - Enable Server-side Apply by default (https://github.com/pulumi/pulumi-kubernetes/pull/2398)
-- Remove deprecated enableDryRun provider flag (https://github.com/pulumi/pulumi-kubernetes/pull/2400)
-- Remove deprecated helm/v2 SDK (https://github.com/pulumi/pulumi-kubernetes/pull/2396)
-- Remove deprecated enableReplaceCRD provider flag (https://github.com/pulumi/pulumi-kubernetes/pull/2402)
-- Drop support for Kubernetes clusters older than v1.13 (https://github.com/pulumi/pulumi-kubernetes/pull/2414)
+- Automatically fall back to client-side preview if server-side preview fails (https://github.com/pulumi/pulumi-kubernetes/pull/2419)
+- Drop support for legacy pulumi.com/initialApiVersion annotation (https://github.com/pulumi/pulumi-kubernetes/pull/2443)
+- Overhaul logic for resource diffing (https://github.com/pulumi/pulumi-kubernetes/pull/2445)
+    - Drop usage of the "kubectl.kubernetes.io/last-applied-configuration" annotation.
+    - Compute preview diffs using resource inputs rather than making a dry-run API call.
+    - Automatically update .metadata.managedFields to work with resources that were managed with client-side apply, and later upgraded to use server-side apply.
+    - Fix a bug with the diff calculation so that resource drift is detected accurately after a refresh.
 
 ## 3.30.2 (July 11, 2023)
 

provider/go.mod

@@ -5,6 +5,7 @@ go 1.19
 require (
 	github.com/ahmetb/go-linq v3.0.0+incompatible
 	github.com/evanphx/json-patch v5.6.0+incompatible
+	github.com/fluxcd/pkg/ssa v0.28.1
 	github.com/golang/protobuf v1.5.3
 	github.com/google/gnostic v0.5.7-v3refs
 	github.com/imdario/mergo v0.3.14
@@ -18,11 +19,11 @@ require (
 	google.golang.org/grpc v1.55.0
 	gopkg.in/yaml.v3 v3.0.1
 	helm.sh/helm/v3 v3.12.0
-	k8s.io/api v0.27.1
-	k8s.io/apimachinery v0.27.1
+	k8s.io/api v0.27.2
+	k8s.io/apimachinery v0.27.2
 	k8s.io/cli-runtime v0.27.1
-	k8s.io/client-go v0.27.1
-	k8s.io/kube-openapi v0.0.0-20230308215209-15aac26d736a
+	k8s.io/client-go v0.27.2
+	k8s.io/kube-openapi v0.0.0-20230501164219-8b0f38b5fd1f
 	k8s.io/kubectl v0.27.1
 	sigs.k8s.io/kustomize/api v0.13.2
 	sigs.k8s.io/kustomize/kyaml v0.14.1
@@ -92,7 +93,7 @@ require (
 	github.com/gedex/inflector v0.0.0-20170307190818-16278e9db813 // indirect
 	github.com/go-errors/errors v1.4.2 // indirect
 	github.com/go-gorp/gorp/v3 v3.0.5 // indirect
-	github.com/go-logr/logr v1.2.3 // indirect
+	github.com/go-logr/logr v1.2.4 // indirect
 	github.com/go-openapi/jsonpointer v0.19.6 // indirect
 	github.com/go-openapi/jsonreference v0.20.1 // indirect
 	github.com/go-openapi/swag v0.22.3 // indirect
@@ -168,10 +169,10 @@ require (
 	github.com/pierrec/lz4 v2.6.1+incompatible // indirect
 	github.com/pkg/term v1.1.0 // indirect
 	github.com/pmezard/go-difflib v1.0.0 // indirect
-	github.com/prometheus/client_golang v1.14.0 // indirect
-	github.com/prometheus/client_model v0.3.0 // indirect
-	github.com/prometheus/common v0.37.0 // indirect
-	github.com/prometheus/procfs v0.8.0 // indirect
+	github.com/prometheus/client_golang v1.15.1 // indirect
+	github.com/prometheus/client_model v0.4.0 // indirect
+	github.com/prometheus/common v0.42.0 // indirect
+	github.com/prometheus/procfs v0.9.0 // indirect
 	github.com/rivo/uniseg v0.4.4 // indirect
 	github.com/rogpeppe/go-internal v1.10.0 // indirect
 	github.com/rubenv/sql-migrate v1.3.1 // indirect
@@ -201,13 +202,13 @@ require (
 	go.uber.org/atomic v1.9.0 // indirect
 	gocloud.dev v0.27.0 // indirect
 	gocloud.dev/secrets/hashivault v0.27.0 // indirect
-	golang.org/x/net v0.8.0 // indirect
+	golang.org/x/net v0.10.0 // indirect
 	golang.org/x/oauth2 v0.6.0 // indirect
-	golang.org/x/sync v0.1.0 // indirect
-	golang.org/x/sys v0.6.0 // indirect
-	golang.org/x/term v0.6.0 // indirect
+	golang.org/x/sync v0.2.0 // indirect
+	golang.org/x/sys v0.8.0 // indirect
+	golang.org/x/term v0.8.0 // indirect
 	golang.org/x/text v0.9.0 // indirect
-	golang.org/x/time v0.0.0-20220722155302-e5dcc9cfc0b9 // indirect
+	golang.org/x/time v0.3.0 // indirect
 	golang.org/x/xerrors v0.0.0-20220907171357-04be3eba64a2 // indirect
 	google.golang.org/api v0.110.0 // indirect
 	google.golang.org/appengine v1.6.7 // indirect
@@ -217,9 +218,9 @@ require (
 	gopkg.in/square/go-jose.v2 v2.6.0 // indirect
 	gopkg.in/warnings.v0 v0.1.2 // indirect
 	gopkg.in/yaml.v2 v2.4.0 // indirect
-	k8s.io/apiextensions-apiserver v0.27.1 // indirect
-	k8s.io/apiserver v0.27.1 // indirect
-	k8s.io/component-base v0.27.1 // indirect
+	k8s.io/apiextensions-apiserver v0.27.2 // indirect
+	k8s.io/apiserver v0.27.2 // indirect
+	k8s.io/component-base v0.27.2 // indirect
 	k8s.io/klog/v2 v2.90.1 // indirect
 	k8s.io/utils v0.0.0-20230220204549-a5ecb0141aa5 // indirect
 	oras.land/oras-go v1.2.2 // indirect
@@ -257,6 +258,7 @@ require (
 	github.com/docker/cli v20.10.21+incompatible // indirect
 	github.com/edsrzf/mmap-go v1.1.0 // indirect
 	github.com/emicklei/go-restful/v3 v3.10.1 // indirect
+	github.com/evanphx/json-patch/v5 v5.6.0 // indirect
 	github.com/go-git/gcfg v1.5.0 // indirect
 	github.com/go-git/go-billy/v5 v5.4.0 // indirect
 	github.com/go-git/go-git/v5 v5.6.0 // indirect
@@ -287,6 +289,8 @@ require (
 	go.opentelemetry.io/otel v1.14.0 // indirect
 	go.opentelemetry.io/otel/trace v1.14.0 // indirect
 	golang.org/x/mod v0.10.0 // indirect
-	golang.org/x/tools v0.7.0 // indirect
+	golang.org/x/tools v0.9.1 // indirect
 	lukechampine.com/frand v1.4.2 // indirect
+	sigs.k8s.io/cli-utils v0.34.0 // indirect
+	sigs.k8s.io/controller-runtime v0.15.0 // indirect
 )