Bug in Thanos chart v1.9.0 for HPA k8s 1.22

[aydosman]

Like #563 we are receiving an autoscaling error occurs when trying to install Thanos chart v1.9.0 using k8s 1.22

Values example

compact:
  enabled: true
logFormat: json
logLevel: info
query:
  autoscaling:
    enabled: true
    maxReplicas: 3
    minReplicas: 1
    targetCPUUtilizationPercentage: 80
queryFrontend:
  autoscaling:
    enabled: true
    maxReplicas: 3
    minReplicas: 1
    targetCPUUtilizationPercentage: 80
  enabled: true
rule:
  enabled: true

Reproduction steps on multiple versions

• 1.22

kind create cluster --image kindest/node:v1.22.15
kubectl config use-context kind-kind
helm repo add stevehipwell https://stevehipwell.github.io/helm-charts/
helm repo update
helm upgrade --install --namespace default --values ./values.yaml my-release stevehipwell/thanos --version "1.7.0"
helm upgrade --install --namespace default --values ./values.yaml my-release stevehipwell/thanos --version "1.8.0"
helm upgrade --install --namespace default --values ./values.yaml my-release stevehipwell/thanos --version "1.9.0" # *FAILS*
kind delete cluster

Failure
Error: UPGRADE FAILED: error validating "": error validating data: ValidationError(HorizontalPodAutoscaler.spec.metrics[0].resource): unknown field "target" in io.k8s.api.autoscaling.v2beta1.ResourceMetricSource

• 1.23

kind create cluster --image kindest/node:v1.23.13
kubectl config use-context kind-kind
helm repo add stevehipwell https://stevehipwell.github.io/helm-charts/
helm repo update
helm upgrade --install --namespace default --values ./values.yaml my-release stevehipwell/thanos --version "1.7.0"
helm upgrade --install --namespace default --values ./values.yaml my-release stevehipwell/thanos --version "1.8.0" # *FAILS*
helm upgrade --install --namespace default --values ./values.yaml my-release stevehipwell/thanos --version "1.9.0"
kind delete cluster

Failure
Error: UPGRADE FAILED: error validating "": error validating data: [ValidationError(HorizontalPodAutoscaler.spec.metrics[0].resource): unknown field "targetAverageUtilization" in io.k8s.api.autoscaling.v2.ResourceMetricSource, ValidationError(HorizontalPodAutoscaler.spec.metrics[0].resource): missing required field "target" in io.k8s.api.autoscaling.v2.ResourceMetricSource]

• 1.24

kind create cluster --image kindest/node:v1.24.7
kubectl config use-context kind-kind
helm repo add stevehipwell https://stevehipwell.github.io/helm-charts/
helm repo update
helm upgrade --install --namespace default --values ./values.yaml my-release stevehipwell/thanos --version "1.7.0"
helm upgrade --install --namespace default --values ./values.yaml my-release stevehipwell/thanos --version "1.8.0" # *FAILS*
helm upgrade --install --namespace default --values ./values.yaml my-release stevehipwell/thanos --version "1.9.0"
kind delete cluster

Failure
Error: UPGRADE FAILED: error validating "": error validating data: [ValidationError(HorizontalPodAutoscaler.spec.metrics[0].resource): unknown field "targetAverageUtilization" in io.k8s.api.autoscaling.v2.ResourceMetricSource, ValidationError(HorizontalPodAutoscaler.spec.metrics[0].resource): missing required field "target" in io.k8s.api.autoscaling.v2.ResourceMetricSource]

• 1.25

kind create cluster --image kindest/node:v1.25.3
kubectl config use-context kind-kind
helm repo add stevehipwell https://stevehipwell.github.io/helm-charts/
helm repo update
helm upgrade --install --namespace default --values ./values.yaml my-release stevehipwell/thanos --version "1.7.0"
helm upgrade --install --namespace default --values ./values.yaml my-release stevehipwell/thanos --version "1.8.0" #`FAILS`
helm upgrade --install --namespace default --values ./values.yaml my-release stevehipwell/thanos --version "1.9.0"

Failure
Error: unable to build kubernetes objects from release manifest: error validating "": error validating data: [ValidationError(HorizontalPodAutoscaler.spec.metrics[0].resource): unknown field "targetAverageUtilization" in io.k8s.api.autoscaling.v2.ResourceMetricSource, ValidationError(HorizontalPodAutoscaler.spec.metrics[0].resource): missing required field "target" in io.k8s.api.autoscaling.v2.ResourceMetricSource]

[stevehipwell]

For K8s v1.23+ version v1.9.0 of the chart works correctly but v1.8.0 is broken. For versions below K8s v1.23 I incorrectly changed the API version for PDB instead of HPA which is why it wasn't working. I've tested v1.9.1 with your values using kubeconform (you can use https://validkube.com/ for this) and this is now working correctly.