helm upgrade -i doesn't update validatingwebhookconfigurations

[kramerul]

Output of helm version: 3.5.0 (and also 3.3.4)

Output of kubectl version:

Client Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.6", GitCommit:"dff82dc0de47299ab66c83c626e08b245ab19037", GitTreeState:"clean", BuildDate:"2020-07-16T00:04:31Z", GoVersion:"go1.14.4", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.14", GitCommit:"89182bdd065fbcaffefec691908a739d161efc03", GitTreeState:"clean", BuildDate:"2020-12-18T12:02:35Z", GoVersion:"go1.13.15", Compiler:"gc", Platform:"linux/amd64"}

Cloud Provider/Platform (AKS, GKE, Minikube etc.): GKE

When updating an existing helm installation the validatingwebhookconfigurations are not updated.

We are trying to deploy the following diff to an existing installation

 apiVersion: admissionregistration.k8s.io/v1beta1
 kind: ValidatingWebhookConfiguration
 metadata:
   name: test
 webhooks:
-{{- range tuple "v1alpha8/cluster" "v1alpha9/cluster"}}
+{{- range tuple "v1alpha8/cluster" "v1alpha9/cluster" "v1alpha10/cluster"}}
 {{- $parts := split "/" . }}
 {{- $version := $parts._0 }}
 {{- $kind := $parts._1 }}
   - admissionReviewVersions:
       - "v1beta1"
     clientConfig:
       service:
         name: test
         namespace: default
         path: "/validate-environment-sapcloud-io-{{$version}}-{{$kind}}"
     failurePolicy: "Fail"
     name: "validate-{{$kind}}.environment.sapcloud.io"
     rules:
       - apiGroups:
           - "environment.sapcloud.io"
         apiVersions:
           - "{{ $version }}"
         operations:
           - "CREATE"
           - "UPDATE"
         resources:
           - "{{$kind}}s"
         scope: "Namespaced"
     sideEffects: "None"
     timeoutSeconds: 10
 {{- end }}

The helm logs look quite promising

$ helm --debug upgrade -i test .
WARNING: "kubernetes-charts.storage.googleapis.com" is deprecated for "stable" and will be deleted Nov. 13, 2020.
WARNING: You should switch to "https://charts.helm.sh/stable" via:
WARNING: helm repo add "stable" "https://charts.helm.sh/stable" --force-update
history.go:56: [debug] getting history for release test
upgrade.go:123: [debug] preparing upgrade for test
upgrade.go:131: [debug] performing update for test
upgrade.go:303: [debug] creating upgraded release for test
client.go:201: [debug] checking 1 resources for changes
upgrade.go:138: [debug] updating status for upgraded release for test
Release "test" has been upgraded. Happy Helming!
NAME: test
LAST DEPLOYED: Thu Jan 28 10:24:53 2021
NAMESPACE: default
STATUS: deployed
REVISION: 7
TEST SUITE: None
USER-SUPPLIED VALUES:
{}

COMPUTED VALUES:
{}

HOOKS:
MANIFEST:
---
# Source: test/templates/webhooks.yaml
apiVersion: admissionregistration.k8s.io/v1beta1
kind: ValidatingWebhookConfiguration
metadata:
  name: test
webhooks:
  - admissionReviewVersions:
      - "v1beta1"
    clientConfig:
      service:
        name: test
        namespace: default
        path: "/validate-environment-sapcloud-io-v1alpha8-cluster"
    failurePolicy: "Fail"
    name: "validate-cluster.environment.sapcloud.io"
    rules:
      - apiGroups:
          - "environment.sapcloud.io"
        apiVersions:
          - "v1alpha8"
        operations:
          - "CREATE"
          - "UPDATE"
        resources:
          - "clusters"
        scope: "Namespaced"
    sideEffects: "None"
    timeoutSeconds: 10
  - admissionReviewVersions:
      - "v1beta1"
    clientConfig:
      service:
        name: test
        namespace: default
        path: "/validate-environment-sapcloud-io-v1alpha9-cluster"
    failurePolicy: "Fail"
    name: "validate-cluster.environment.sapcloud.io"
    rules:
      - apiGroups:
          - "environment.sapcloud.io"
        apiVersions:
          - "v1alpha9"
        operations:
          - "CREATE"
          - "UPDATE"
        resources:
          - "clusters"
        scope: "Namespaced"
    sideEffects: "None"
    timeoutSeconds: 10
  - admissionReviewVersions:
      - "v1beta1"
    clientConfig:
      service:
        name: test
        namespace: default
        path: "/validate-environment-sapcloud-io-v1alpha10-cluster"
    failurePolicy: "Fail"
    name: "validate-cluster.environment.sapcloud.io"
    rules:
      - apiGroups:
          - "environment.sapcloud.io"
        apiVersions:
          - "v1alpha10"
        operations:
          - "CREATE"
          - "UPDATE"
        resources:
          - "clusters"
        scope: "Namespaced"
    sideEffects: "None"
    timeoutSeconds: 10

But the result is not the expected one

$ kubectl get validatingwebhookconfigurations.admissionregistration.k8s.io test -o yaml | grep v1alpha
      path: /validate-environment-sapcloud-io-v1alpha8-cluster
    - v1alpha8
      path: /validate-environment-sapcloud-io-v1alpha9-cluster
    - v1alpha9

[kramerul]

It seems to be a kubernetes problem

helm template . | kubectl apply -f - also doesn't work.

[kramerul]

See kubernetes/kubernetes#98525 for solution