Container Cluster stuck in non-ready state because channel update rejected

[jlewi]

Describe the bug
A clear and concise description of what the bug is.

ConfigConnector Version
Run the following command to get the current ConfigConnector version

kubectl get ns cnrm-system -o jsonpath='{.metadata.annotations.cnrm\.cloud\.google\.com/version}' 

cnrm.cloud.google.com/version: 1.9.1

To Reproduce
Steps to reproduce the behavior:

1. Create a ContainerCluster CNRM resource setting the release channel for the cluster to get
   the status

2. Apply the resource to create the cluster

3. Reapply the resource

4. Container cluster reports

     - lastTransitionTime: "2020-05-29T00:15:38Z"
      message: 'Update call failed: the desired mutation for the following field(s)
        is invalid: [releaseChannel.0.Channel]'
      reason: UpdateFailed
      status: "False"
      type: Ready
   

The release channel shouldn't be changing.

I suspect this an issue in the update logic since there are some restrictions about mutations to
release channel
https://cloud.google.com/kubernetes-engine/docs/concepts/release-channels

YAML snippets:

apiVersion: container.cnrm.cloud.google.com/v1beta1
kind: ContainerCluster
metadata:
  clusterName: gcp-private-dev/us-central1/jl-0601
  labels:
    kf-name: gcp-private-0527
    mesh_id: gcp-private-dev_us-central1_jl-0601
  name: jl-0601
  namespace: gcp-private-dev
spec:
  clusterAutoscaling:
    autoProvisioningDefaults:
      oauthScopes:
      - https://www.googleapis.com/auth/logging.write
      - https://www.googleapis.com/auth/monitoring
      - https://www.googleapis.com/auth/devstorage.read_only
      serviceAccountRef:
        name: jl-0601-vm
    enabled: true
    resourceLimits:
    - maximum: 128
      resourceType: cpu
    - maximum: 2000
      resourceType: memory
    - maximum: 16
      resourceType: nvidia-tesla-k80
  initialNodeCount: 2
  ipAllocationPolicy:
    clusterSecondaryRangeName: pods
    createSubnetwork: false
    servicesSecondaryRangeName: services
    useIpAliases: true
  location: us-central1
  loggingService: logging.googleapis.com/kubernetes
  monitoringService: monitoring.googleapis.com/kubernetes
  networkRef:
    name: jl-0601
  nodeConfig:
    machineType: n1-standard-8
    metadata:
      disable-legacy-endpoints: "true"
    oauthScopes:
    - https://www.googleapis.com/auth/logging.write
    - https://www.googleapis.com/auth/monitoring
    - https://www.googleapis.com/auth/devstorage.read_only
    serviceAccountRef:
      name: jl-0601-vm
    workloadMetadataConfig:
      nodeMetadata: GKE_METADATA_SERVER
  privateClusterConfig:
    enablePrivateEndpoint: false
    enablePrivateNodes: true
    masterIpv4CidrBlock: 172.16.0.32/28
  releaseChannel:
    channel: stable
  subnetworkRef:
    name: jl-0601
  workloadIdentityConfig:
    identityNamespace: gcp-private-dev.svc.id.goog

[kibbles-n-bytes]

Hey @jlewi , the underlying API's canonical form for the release channel is all uppercase: https://cloud.google.com/kubernetes-engine/docs/reference/rest/v1beta1/projects.locations.clusters#channel

Curiously, the underlying API accepts the lowercase form as well, but further GET requests return the uppercase form, which we currently don't suppress a diff for. If you create your cluster with channel: STABLE instead, you should see this issue go away.

[jlewi]

Thanks that's super helpful I will fix that on our end.

Feel free to close this issue unless you want to leave open to continue to track handling the diff more gracefully.