Register K3s CRDs with schema

[brandond]

K3s tracking issue for:

• Bool in HelmChartSpec.items.spec.set can stall controller queue processing helm-controller#172
• helmchart resources no longer spawn helm-install jobs after update rancher/rke2#3849

We are registering the CRDs without any schema, which allows users to create resources with invalid data. The problem wouldn't show up until the resource was deserialized by the controller, at which point it would stop processing events from the watch channel because it was stuck on the event with the bad resource.

Example: create the following resource (with one of the set keys using an incorrect value of boolean type)

apiVersion: v1
kind: Namespace
metadata:
  name: cert-manager
---
apiVersion: helm.cattle.io/v1
kind: HelmChart
metadata:
  namespace: kube-system
  name: cert-manager
spec:
  targetNamespace: cert-manager
  version: v1.9.1
  chart: cert-manager
  repo: https://charts.jetstack.io
  set:
    installCRDs: true
    webhook.timeoutSeconds: 4

You should see the following error:
The HelmChart "cert-manager" is invalid: spec.set.installCRDs: Invalid value: "boolean": spec.set.installCRDs in body must be of type integer,string: "boolean"

The HelmChart should be accepted when the value is properly quoted as "true"

[VestigeJ]

##Environment Details
Confirmed existing behavior using VERSION=v1.27.1+k3s1
Confirmed new validation using COMMIT=8f450bafe1cad0e962e521d56eb74a38a73722c7

Infrastructure

• Cloud

Node(s) CPU architecture, OS, and version:

Linux 5.14.21-150400.24.11-default x86_64 GNU/Linux 
PRETTY_NAME="SUSE Linux Enterprise Server 15 SP4"

Cluster Configuration:

NAME              STATUS   ROLES                       AGE     VERSION
ip-2-2-2-28       Ready    control-plane,etcd,master   8m47s   v1.27.1+k3s1 

Config.yaml:

write-kubeconfig-mode: 644
debug: true
token: saucewatch
selinux: true
protect-kernel-defaults: true
cluster-init: true

Reproduced behavior existing behavior

$ curl https://get.k3s.io --output install-"k3s".sh
$ sudo chmod +x install-"k3s".sh
$ sudo groupadd --system etcd && sudo useradd -s /sbin/nologin --system -g etcd etcd
$ sudo modprobe ip_vs_rr
$ sudo modprobe ip_vs_wrr
$ sudo modprobe ip_vs_sh
$ sudo printf "on_oovm.panic_on_oom=0 \nvm.overcommit_memory=1 \nkernel.panic=10 \nkernel.panic_ps=1 \nkernel.panic_on_oops=1 \n" > ~/60-rke2-cis.conf or ~/90-kubelet.conf
$ sudo cp 60-rke2-cis.conf /etc/sysctl.d/ or 90-kubelet.conf
$ sudo systemctl restart systemd-sysctl
$ sudo INSTALL_K3S_VERSION=v1.27.1+k3s1 INSTALL_K3S_EXEC=server ./install-k3s.sh 
$ set_kubefig //export KUBECONFIG=/etc/rancher/k3s/k3s.yaml
$ kgn //kubectl get nodes
$ kgp -A //kubectl get pods -A
$ kg crd -A //kubectl get crd -A
$ vim bad_boolean.yaml //see full file below
$ k apply -f bad_boolean.yaml 
$ get_report //generate this template

Results:
The malformed file is accepted and not rejected due to a schema mismatch
$ k apply -f bad_boolean.yaml

namespace/cert-manager created

Validated schema rejects malformed yaml

Validation Steps

$ curl https://get.k3s.io --output install-"k3s".sh
$ sudo chmod +x install-"k3s".sh
$ sudo groupadd --system etcd && sudo useradd -s /sbin/nologin --system -g etcd etcd
$ sudo modprobe ip_vs_rr
$ sudo modprobe ip_vs_wrr
$ sudo modprobe ip_vs_sh
$ sudo printf "on_oovm.panic_on_oom=0 \nvm.overcommit_memory=1 \nkernel.panic=10 \nkernel.panic_ps=1 \nkernel.panic_on_oops=1 \n" > ~/60-rke2-cis.conf or ~/90-kubelet.conf
$ sudo cp 60-rke2-cis.conf /etc/sysctl.d/ or 90-kubelet.conf
$ sudo systemctl restart systemd-sysctl
$ sudo INSTALL_K3S_COMMIT=8f450bafe1cad0e962e521d56eb74a38a73722c7 INSTALL_K3S_EXEC=server ./install-k3s.sh 
$ set_kubefig //export KUBECONFIG=/etc/rancher/k3s/k3s.yaml
$ kgn //kubectl get nodes
$ kgp -A //kubectl get pods -A
$ kg crd -A //kubectl get crd -A
$ vim bad_boolean.yaml //see full file below
$ vim good_boolean.yaml //see full file below attn: true quote encapsulated
$ k apply -f bad_boolean.yaml //observe results below
$ k apply -f good_boolean.yaml //observe results below

Results:

$ k apply -f bad_boolean.yaml

namespace/cert-manager unchanged
The HelmChart "cert-manager" is invalid: spec.set.installCRDs: Invalid value: "boolean": spec.set.installCRDs in body must be of type integer,string: "boolean"

$ k apply -f good_boolean.yaml

namespace/cert-manager unchanged
helmchart.helm.cattle.io/cert-manager created

Additional context / logs:

$ cat bad_boolean.yaml

apiVersion: v1
kind: Namespace
metadata:
  name: cert-manager
---
apiVersion: helm.cattle.io/v1
kind: HelmChart
metadata:
  namespace: kube-system
  name: cert-manager
spec:
  targetNamespace: cert-manager
  version: v1.9.1
  chart: cert-manager
  repo: https://charts.jetstack.io
  set:
    installCRDs: true
    webhook.timeoutSeconds: 4

$ cat good_boolean.yaml

apiVersion: v1
kind: Namespace
metadata:
  name: cert-manager
---
apiVersion: helm.cattle.io/v1
kind: HelmChart
metadata:
  namespace: kube-system
  name: cert-manager
spec:
  targetNamespace: cert-manager
  version: v1.9.1
  chart: cert-manager
  repo: https://charts.jetstack.io
  set:
    installCRDs: "true"
    webhook.timeoutSeconds: 4