helm chart not compatible with k8s v1.22

[qria]

Describe the bug

Running Install Seldon Core fails in k8s v1.22. Specifically, it fails at helm install.

It's probably caused by deprecation of rbac v1beta1.

To reproduce

1. Install k8s 1.22 with minikube
2. Run install script provided by quickstart documentation.

kubectl create namespace seldon-system

helm install seldon-core seldon-core-operator \
    --repo https://storage.googleapis.com/seldon-charts \
    --set usageMetrics.enabled=true \
    --namespace seldon-system \
    --set istio.enabled=true

Expected behaviour

It should install seldon-core.

Environment

• Cloud provider: Minikube
• kubectl version

Client Version: version.Info{Major:"1", Minor:"22", GitVersion:"v1.22.1", GitCommit:"632ed300f2c34f6d6d15ca4cef3d3c7073412212", GitTreeState:"clean", BuildDate:"2021-08-19T15:38:26Z", GoVersion:"go1.16.6", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"22", GitVersion:"v1.22.1", GitCommit:"632ed300f2c34f6d6d15ca4cef3d3c7073412212", GitTreeState:"clean", BuildDate:"2021-08-19T15:39:34Z", GoVersion:"go1.16.7", Compiler:"gc", Platform:"linux/amd64"}

Error Log

❯ helm install seldon-core seldon-core-operator \
    --repo https://storage.googleapis.com/seldon-charts \
    --set usageMetrics.enabled=true \
    --namespace seldon-system \
    --set istio.enabled=true
WARNING: Kubernetes configuration file is group-readable. This is insecure. Location: /Users/qria/.kube/config
WARNING: Kubernetes configuration file is world-readable. This is insecure. Location: /Users/qria/.kube/config
Error: INSTALLATION FAILED: unable to build kubernetes objects from release manifest: [unable to recognize "": no matches for kind "ClusterRole" in version "rbac.authorization.k8s.io/v1beta1", unable to recognize "": no matches for kind "ClusterRoleBinding" in version "rbac.authorization.k8s.io/v1beta1", unable to recognize "": no matches for kind "ValidatingWebhookConfiguration" in version "admissionregistration.k8s.io/v1beta1"]

[sleebapaul]

Is there any update on this one? Facing the same issue. Anyone could suggest a compatible version, if it is just for v1.22.2?

[axsaucedo]

@sleebapaul this issue would generally encompass support for Kubernetes 1.22 - currently our e2e tests are running in 1.21, so we will be looking at making sure that on one side the helm charts are updated, but more importantly that all features work in 1.22. We are looking for this issue to be addressed for our upcoming 1.12 release of Seldon Core in 6-8 weeks. For now the suggestion would be to use k8s 1.18 >= and <= 1.21

[sleebapaul]

@axsaucedo Still no luck.

kubectl version

Client Version: version.Info{Major:"1", Minor:"21", GitVersion:"v1.21.5", GitCommit:"aea7bbadd2fc0cd689de94a54e5b7b758869d691", GitTreeState:"clean", BuildDate:"2021-09-15T21:10:45Z", GoVersion:"go1.16.8", Compiler:"gc", Platform:"darwin/arm64"}
Server Version: version.Info{Major:"1", Minor:"22", GitVersion:"v1.22.2", GitCommit:"8b5a19147530eaac9476b0ab82980b4088bbc1b2", GitTreeState:"clean", BuildDate:"2021-09-15T21:32:41Z", GoVersion:"go1.16.8", Compiler:"gc", Platform:"linux/arm64"}

minikube version

minikube version: v1.23.2
commit: 0a0ad764652082477c00d51d2475284b5d39ceed

helm version

version.BuildInfo{Version:"v3.7.0", GitCommit:"eeac83883cb4014fe60267ec6373570374ce770b", GitTreeState:"clean", GoVersion:"go1.17"}

helm install seldon-core seldon-core-operator --repo https://storage.googleapis.com/seldon-charts --set istio.enabled=true --set usageMetrics.enabled=true --namespace seldon-system

Error: INSTALLATION FAILED: unable to build kubernetes objects from release manifest: [unable to recognize "": no matches for kind "ClusterRole" in version "rbac.authorization.k8s.io/v1beta1", unable to recognize "": no matches for kind "ClusterRoleBinding" in version "rbac.authorization.k8s.io/v1beta1", unable to recognize "": no matches for kind "ValidatingWebhookConfiguration" in version "admissionregistration.k8s.io/v1beta1"]

[qria]

@sleebapaul Looks like your k8s server version is still v1.22, downgrade to v1.21 because that's the latest support version. You can start downgraded k8s cluster with minikube start --kubernetes-version=v1.21.4

[sleebapaul]

That worked. Thanks a lot @qria .

[rokosbasilisk]

i am still facing an issue with installing seldon via helm-api
this is the error message:-
UNKNOWN: validation failed: unable to recognize "": no matches for kind "ValidatingWebhookConfiguration" in version "admissionregistration.k8s.io/v1beta1

kubectl version is v1.22.1

[metanouvelle]

I have the same issue as @rokosbasilisk .

kubectl version

Client Version: version.Info{Major:"1", Minor:"22", GitVersion:"v1.22.4", GitCommit:"b695d79d4f967c403a96986f1750a35eb75e75f1", GitTreeState:"clean", BuildDate:"2021-11-17T15:48:33Z", GoVersion:"go1.16.10", Compiler:"gc", `Platform:"darwin/arm64"}

Server Version: version.Info{Major:"1", Minor:"21+", GitVersion:"v1.21.5-eks-bc4871b", GitCommit:"5236faf39f1b7a7dabea8df12726f25608131aa9", GitTreeState:"clean", BuildDate:"2021-10-29T23:32:16Z", GoVersion:"go1.16.8", Compiler:"gc", Platform:"linux/amd64"}

[metanouvelle]

Besides Server Version, do we also need to downgrade Client version to v1.21? @qria

[qria]

I don't recall but probably not.

[AmbroxMr]

Hello, the Kubernetes version 1.21 end of life is approaching (2022-06-28). Will you eventually update the analytics chart with newer Grafana and Prometheus chart versions?

[ukclivecox]

We have a PR #4038 which moves analytics to use the prometheus operator instead.

[Al4DIN]

Hello, I've the same issue while trying to install seldon-analytics (Grafana & Prometheus)

This is my kubectl version

Client Version: version.Info{Major:"1", Minor:"22", GitVersion:"v1.22.5", GitCommit:"5c99e2ac2ff9a3c549d9ca665e7bc05a3e18f07e", GitTreeState:"clean", BuildDate:"2021-12-16T08:38:33Z", GoVersion:"go1.16.12", Compiler:"gc", Platform:"windows/amd64"}
Server Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.4", GitCommit:"e6c093d87ea4cbb530a7b2ae91e54c0842d8308a", GitTreeState:"clean", BuildDate:"2022-03-06T21:32:53Z", GoVersion:"go1.17.7", Compiler:"gc", Platform:"linux/amd64"}

And I'm using Kind, and this is my version kind v0.12.0 go1.17.8 windows/amd64

This is the error :

Error: unable to build kubernetes objects from release manifest: [unable to recognize "": no matches for kind "ClusterRole" in version "rbac.authorization.k8s.io/v1beta1", unable to recognize "": no matches for kind "ClusterRoleBinding" in version "rbac.authorization.k8s.io/v1beta1", unable to recognize "": no matches for kind "Role" in version "rbac.authorization.k8s.io/v1beta1", unable to recognize "": no matches for kind "RoleBinding" in version "rbac.authorization.k8s.io/v1beta1"]

So except downgrading my kubectl version I can't fix it?