DaemonSetPatch taking ownership of fields unexpectedly upon an update operation with ignoreChanges set

[zbuchheit]

What happened?

While using the Pulumi DaemonSetPatch resource, I encountered a bug where having ignoreChanges set caused unexpected behavior. Specifically, when this parameter was set, the Pulumi process took unexpected ownership of resources during an update operation. It did not occur on the initial resource creation, but on an update operation once a field was changed.

I had ignoreChanges set to spec.template.metadata and spec.selector and witnessed the pulumi-kubernetes manager take ownership of spec.template.metadata but not spec.selector

Example

Code Example

		daemonSet, err := v1.NewDaemonSetPatch(ctx, "k8s-daemonset-patch", &v1.DaemonSetPatchArgs{
					Metadata: metav1.ObjectMetaPatchArgs{
						Name:      pulumi.String("kube-proxy"),
						Namespace: pulumi.String("kube-system"),
						Annotations: pulumi.StringMap{
							"pulumi.com/patchForce":        pulumi.String("true"),
						},
					},
					Spec: v1.DaemonSetSpecPatchArgs{
						Template: corev1.PodTemplateSpecPatchArgs{
							Spec: corev1.PodSpecPatchArgs{
								Containers: corev1.ContainerPatchArray{
									corev1.ContainerPatchArgs{
										Name:  pulumi.String("kube-proxy"),
										Image: pulumi.String("602401143452.dkr.ecr.us-west-2.amazonaws.com/eks/kube-proxy:v1.24.17-minimal-eksbuild.2"), //Change this to for update
									},
								},
							},
						},
					},
				}, pulumi.Provider(kubeProvider),
				pulumi.IgnoreChanges([]string{"spec.template.metadata", "spec.selector"}), 
				pulumi.RetainOnDelete(true), 
			)

Resulting Config Information

The kube-proxy config on resource creation.
The kube-proxy config after an update with ignoreChanges set.

Output of pulumi about

CLI          
Version      3.92.0
Go Version   go1.21.3
Go Compiler  gc

Plugins
NAME        VERSION
go          unknown
kubernetes  4.5.5

Host     
OS       darwin
Version  14.0
Arch     arm64

This project is written in go: executable='/opt/homebrew/bin/go' version='go version go1.21.0 darwin/arm64'

Current Stack: zbuchheit-pulumi-corp/go-daemon-patch/dev

TYPE                               URN
pulumi:pulumi:Stack                urn:pulumi:dev::go-daemon-patch::pulumi:pulumi:Stack::go-daemon-patch-dev
pulumi:providers:pulumi            urn:pulumi:dev::go-daemon-patch::pulumi:providers:pulumi::default
pulumi:pulumi:StackReference       urn:pulumi:dev::go-daemon-patch::pulumi:pulumi:StackReference::zbuchheit-pulumi-corp/aws-ts-eks/dev
pulumi:providers:kubernetes        urn:pulumi:dev::go-daemon-patch::pulumi:providers:kubernetes::eks-kubernetes
kubernetes:apps/v1:DaemonSetPatch  urn:pulumi:dev::go-daemon-patch::kubernetes:apps/v1:DaemonSetPatch::k8s-daemonset-patch


Found no pending operations associated with dev

Backend        
Name           pulumi.com
URL            https://app.pulumi.com/zbuchheit-pulumi-corp
User           zbuchheit-pulumi-corp
Organizations  zbuchheit-pulumi-corp
Token type     personal

Dependencies:
NAME                                        VERSION
github.com/pulumi/pulumi-kubernetes/sdk/v4  4.5.5
github.com/pulumi/pulumi/sdk/v3             3.92.0

Pulumi locates its logs in /var/folders/lh/l71cdh810xb33t0jc7qmt5_80000gn/T/ by default

Additional context

I believe it might be slightly related to the described behavior here: #2639 (comment)

Contributing

Vote on this issue by adding a 👍 reaction.
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

[rquitales]

Thanks for the detailed logs. It looks like we might need to harden our fieldManager path traversal/matching logic.

[arwilczek90]

I can also confirm that this happens on the image field for almost any workload which is a very dangerous bug if you are not using pulumi to deploy image changes.