Unexpected CR conversion behavior

[AntonAleksandrov13]

Discussed in #3704

^{Originally posted by AntonAleksandrov13 December 8, 2023}
Hi,
I have implemented conversion webhook for my CRD per guide from kubebuilder book.
I have dedicated v1beta1 version as Hub and v1alpha1 version as Spoke. Below, you can find code for conversion between the versions:

api/v1beta1/teamcity_conversions.go

package v1beta1

func (*TeamCity) Hub() {}

api/v1alpha/teamcity_conversions.go

// ConvertTo converts this TeamCity to the Hub version (v1beta1).
func (src *TeamCity) ConvertTo(dstRaw conversion.Hub) error {
	dst := dstRaw.(*v1beta1.TeamCity)

	//version specific conversion
	dataDirIndex := 0
	dataDir := src.Spec.PersistentVolumeClaims[dataDirIndex]
	dst.Spec.DataDirVolumeClaim = v1beta1.CustomPersistentVolumeClaim(dataDir)
	//remove data dir from list of pvcs
	trimmedPersistentVolumeClaims := append(src.Spec.PersistentVolumeClaims[:dataDirIndex], src.Spec.PersistentVolumeClaims[dataDirIndex:]...)
	//add other elements into a new type
	var newPersistentVolumeClaimList []v1beta1.CustomPersistentVolumeClaim
	for _, element := range trimmedPersistentVolumeClaims {
		newPersistentVolumeClaimList = append(newPersistentVolumeClaimList, v1beta1.CustomPersistentVolumeClaim(element))
	}
	//save list of persistent volumes into a new object
	dst.Spec.PersistentVolumeClaims = newPersistentVolumeClaimList
	//the rest fields that remain the same
	//object meta
	dst.ObjectMeta = src.ObjectMeta

	//object spec
	dst.Spec.Env = src.Spec.Env
	dst.Spec.Image = src.Spec.Image
	dst.Spec.Requests = src.Spec.Requests
	dst.Spec.Limits = src.Spec.Limits
	dst.Spec.TeamCityServerPort = src.Spec.TeamCityServerPort
	dst.Spec.PodSecurityContext = src.Spec.PodSecurityContext
	dst.Spec.XmxPercentage = src.Spec.XmxPercentage

	dst.Spec.HealthEndpoint = src.Spec.HealthEndpoint
	dst.Spec.ReadinessProbeSettings = src.Spec.ReadinessProbeSettings
	dst.Spec.LivenessProbeSettings = src.Spec.LivenessProbeSettings
	dst.Spec.StartupProbeSettings = src.Spec.StartupProbeSettings

	dst.Spec.InitContainers = src.Spec.InitContainers
	dst.Spec.StartupPropertiesConfig = src.Spec.StartupPropertiesConfig
	dst.Spec.DatabaseSecret = v1beta1.DatabaseSecret(src.Spec.DatabaseSecret)

	//object status
	dst.Status = v1beta1.TeamCityStatus(src.Status)
	return nil
}

// ConvertFrom converts from the Hub version (v1beta1) to this version.
func (dst *TeamCity) ConvertFrom(srcRaw conversion.Hub) error {
	src := srcRaw.(*v1beta1.TeamCity)

	//version specific conversion
	dataDirPersistentVolumeClaim := CustomPersistentVolumeClaim(src.Spec.DataDirVolumeClaim)
	var persistentVolumeClaimList []CustomPersistentVolumeClaim
	persistentVolumeClaimList = append(persistentVolumeClaimList, dataDirPersistentVolumeClaim)
	for _, elem := range src.Spec.PersistentVolumeClaims {
		persistentVolumeClaimList = append(persistentVolumeClaimList, CustomPersistentVolumeClaim(elem))
	}
	//object meta
	dst.ObjectMeta = src.ObjectMeta

	//object spec
	dst.Spec.Env = src.Spec.Env
	dst.Spec.Image = src.Spec.Image
	dst.Spec.Requests = src.Spec.Requests
	dst.Spec.Limits = src.Spec.Limits
	dst.Spec.TeamCityServerPort = src.Spec.TeamCityServerPort
	dst.Spec.PodSecurityContext = src.Spec.PodSecurityContext
	dst.Spec.XmxPercentage = src.Spec.XmxPercentage

	dst.Spec.HealthEndpoint = src.Spec.HealthEndpoint
	dst.Spec.ReadinessProbeSettings = src.Spec.ReadinessProbeSettings
	dst.Spec.LivenessProbeSettings = src.Spec.LivenessProbeSettings
	dst.Spec.StartupProbeSettings = src.Spec.StartupProbeSettings

	dst.Spec.InitContainers = src.Spec.InitContainers
	dst.Spec.StartupPropertiesConfig = src.Spec.StartupPropertiesConfig
	dst.Spec.DatabaseSecret = DatabaseSecret(src.Spec.DatabaseSecret)

	dst.Status = TeamCityStatus(src.Status)

	return nil
}

I have also added +kubebuilder:storageversion annotation in v1beta1 type. In the generated CRD, storage: true is set for v1beta1 version and storage: false for v1alpha1 version.

I believe my only deviation from the tutorial is that I use kustomize to patch CRD and Webhooks configuration to be able to run operator with webhooks locally as was suggested here - #400 (comment)

In the end, my conversion field in CRD looks like this:

  conversion:
    strategy: Webhook
    webhook:
      clientConfig:
        caBundle: <CA_BUNDLE>
        url: https://host.minikube.internal:9443/convert
      conversionReviewVersions:
      - v1

Observed behaviour:

When I try to apply object with version v1beta1, reconciliation is invoked and all is well.

However, when I try to apply object of v1alpha1 version, I can see that first ConvertTo function is invoked(which is expected) and ConversionResponse is returned with a valid object of version v1beta1. No error occurs. Immediately after, ConvertFrom function is invoked which is not expected. That indicates that for some reason conversion from v1beta1 to v1alpha1 is happening. That I cannot exactly explain. Once ConversionResponse is sent with a valid version v1alpha1, nothing happens - no custome object is created(neither v1beta1 nor v1alpha1).
I have inspected kube-api logs and could not find anything.

I have checked for similar issues, but could only find #2109. This does not seems to be my case because conversions back and forth is not repeated a few times. Therefore, I am looking for guidance on this matter.

Kind regards,
Anton