helm3 resource types not available in for APIVersion Capability queries during template command

[skuethe]

Output of helm version:

version.BuildInfo{Version:"v3.4.1", GitCommit:"c4e74854886b2efe3321e185578e6db9be0a6e29", GitTreeState:"clean", GoVersion:"go1.14.11"}

Output of kubectl version:

Client Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.6", GitCommit:"dff82dc0de47299ab66c83c626e08b245ab19037", GitTreeState:"clean", BuildDate:"2020-07-15T16:58:53Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.6", GitCommit:"dff82dc0de47299ab66c83c626e08b245ab19037", GitTreeState:"clean", BuildDate:"2020-07-15T16:51:04Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}

Cloud Provider/Platform (AKS, GKE, Minikube etc.):
on-premise

Introduction
I am currently running into some specific problems deploying helm charts on a kubernetes 1.18.6 cluster. These problems can be broken down to the fact that, helm template somehow does not fetch "resource types" during usage of .Capabilities.APIVersions.Has queries.

Details
Some in-depth details about the problem and my analysis steps:

1. we are using argo cd to apply helm charts to the 1.18.6 Kubernetes cluster. This works just fine, but I had some trouble installing specific charts with enabled ingress option. Specifically f.e. Gitea - see their ingress template
2. All of the failed ingress implementations had something common: they use the if .Capabilities.APIVersions.Has "networking.k8s.io/v1" function to differentiate between 1.19, 1.18 and older Kubernetes API versions and their ingress implementation. In this case, Kubernetes 1.18 API uses networking.k8s.io/v1beta1 and with Kubernetes 1.19 API it is switching to networking.k8s.io/v1
3. Argo CD would show me the "desired manifest" to actually use networking.k8s.io/v1 despite using Kubernetes 1.18 - see
   
4. The problem was found quickly: The command if .Capabilities.APIVersions.Has "networking.k8s.io/v1" IS and SHOULD be true, as 1.18 is supporting networking.k8s.io/v1/NetworkPolicy. Unfortunately this breaks creation of Ingress Objects
5. I then tried to fork and modify the ingress files to include the ressource type in the if call like this: if .Capabilities.APIVersions.Has "networking.k8s.io/v1/Ingress" however this seems to not work with the current helm v3 version. See the example data below

I have stumbled upon this PR which should actually make it possible to use the ressource types, right?
And it also says so in the documentation

Data to recreate

values.yaml:

ingress:
  enabled: true

templates/ingress.yaml:

{{- if .Capabilities.APIVersions.Has "networking.k8s.io/v1/Ingress" -}}
apiVersion: networking.k8s.io/v1
{{- else if .Capabilities.APIVersions.Has "networking.k8s.io/v1beta1/Ingress" -}}
apiVersion: networking.k8s.io/v1beta1
{{- else -}}
apiVersion: extensions/v1beta1
{{- end }}
kind: Ingress

Testing:

$ helm template testingress ./
---
# Source: ingresstest/templates/ingress.yaml
apiVersion: extensions/v1beta1
kind: Ingress

Any thoughts on this? Am I missing something completely here?
I would like to add a PR for the external helm charts to actually use the resource type in their .Capabilities.APIVersions.Has queries, but this obviously requires it to work in the first place :)

I also tried to play around with helm template --api-versions but to be honest, I did not fully understand how to use it. Adding anything there just didn't change the output at all.

[skuethe]

ah sorry, that values.yaml input for re-creating is obsolete. it was from some older testing and not used anymore (as you of course all can see :D)

[bacongobbler]

helm template renders client-side. If you want to fetch your cluster's capabilities, use helm template --validate.

><> helm help template | grep validate
      --disable-openapi-validation   if set, the installation process will not validate rendered templates against the Kubernetes OpenAPI Schema
      --validate                     validate your manifests against the Kubernetes cluster you are currently pointing at. This is the same validation performed on an install

Does that help?

[skuethe]

So it was something I was missing.
Yes, that helped. With that I can request PRs on the external charts :)

Thank you for the quick reply and help!

[mteodori]

thanks, I haven't understood the usage of "--api-versions" as well, as I thought using "--api-versions=v1.17.0" would have produced matching values in .Capabilities.KubeVersion and .Capabilities.APIVersions while I seem to get "v1.18.0" whatever I do

[mteodori]

I think I was confused by this which seems to be wrong https://github.com/rikatz/kubepug#example-of-usage-in-ci-with-github-actions

      - name: Run Kubepug with your Helm Charts Repository
        run: |
          find charts -mindepth 1 -maxdepth 1 -type d | xargs -t -n1 -I% /bin/bash -c '~/bin/helm template % --api-versions ${K8S_TARGET_VERSION} | ~/bin/kubepug --error-on-deprecated --error-on-deleted --k8s-version ${K8S_TARGET_VERSION} --input-file /dev/stdin'

can this be a feature request? getting all the api versions of a specific kubernetes version as an additional --k8s-version flag in helm too?