OpenShift GitOps Operator installation on a OpenShift cluster with Kubernetes v1.24 fails

[bnallapeta]

Describe the bug
OpenShift GitOps Operator installation on a OpenShift cluster using Kubernetes 1.24 fails.
To be more precise, not all the components of the operator are installed. Just the cluster and kam deployments.

$ oc version
Client Version: 4.9.23
Server Version: 4.11.0-0.ci-2022-05-29-145655
**Kubernetes Version: v1.24.0-alpha.3.3613+01aa0f3f6052c9-dirty**

Red Hat OpenShift GitOps
1.5.2 provided by Red Hat Inc.

To Reproduce
Steps to reproduce the behavior:

1. On a OpenShift cluster with the latest 4.11 nightly build (running k8s v1.24), install OpenShift GitOps Operator.

Current behavior

$ oc get all -n openshift-gitops
NAME                           READY   STATUS    RESTARTS   AGE
pod/cluster-75d847dc67-hrl9t   1/1     Running   0          41m
pod/kam-d9fd64b9-qvppn         1/1     Running   0          41m

NAME              TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)             AGE
service/cluster   ClusterIP   172.30.211.107   <none>        8080/TCP            41m
service/kam       ClusterIP   172.30.222.105   <none>        8080/TCP,8443/TCP   41m

NAME                      READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/cluster   1/1     1            1           41m
deployment.apps/kam       1/1     1            1           41m

NAME                                 DESIRED   CURRENT   READY   AGE
replicaset.apps/cluster-75d847dc67   1         1         1       41m
replicaset.apps/kam-d9fd64b9         1         1         1       41m

NAME                           HOST/PORT                                                                         PATH   SERVICES   PORT   TERMINATION        WILDCARD
route.route.openshift.io/kam   kam-openshift-gitops.apps.ci-ln-r4vtbyt-72292.origin-ci-int-gce.dev.rhcloud.com          kam        8443   passthrough/None   None

Expected behavior
All GitOps components should be installed.

$ oc -n openshift-gitops get all
NAME                                                              READY   STATUS    RESTARTS   AGE
pod/cluster-7c4bdc7756-bbn69                                      1/1     Running   0          2m4s
pod/kam-b49647dc5-rz9bf                                           1/1     Running   0          2m4s
pod/openshift-gitops-application-controller-0                     1/1     Running   0          2m2s
pod/openshift-gitops-applicationset-controller-574df4d4d7-mfrtf   1/1     Running   0          2m2s
pod/openshift-gitops-dex-server-67779567cd-stg74                  1/1     Running   0          2m1s
pod/openshift-gitops-redis-7867d74fb4-x7jd6                       1/1     Running   0          2m2s
pod/openshift-gitops-repo-server-8568c8b7f-hvnxw                  1/1     Running   0          2m2s
pod/openshift-gitops-server-78468747f9-v25dg                      1/1     Running   0          2m2s

NAME                                      TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)             AGE
service/cluster                           ClusterIP   172.30.241.15    <none>        8080/TCP            2m4s
service/kam                               ClusterIP   172.30.146.48    <none>        8080/TCP,8443/TCP   2m4s
service/openshift-gitops-dex-server       ClusterIP   172.30.169.60    <none>        5556/TCP,5557/TCP   2m3s
service/openshift-gitops-metrics          ClusterIP   172.30.222.104   <none>        8082/TCP            2m3s
service/openshift-gitops-redis            ClusterIP   172.30.35.35     <none>        6379/TCP            2m3s
service/openshift-gitops-repo-server      ClusterIP   172.30.82.118    <none>        8081/TCP,8084/TCP   2m3s
service/openshift-gitops-server           ClusterIP   172.30.170.231   <none>        80/TCP,443/TCP      2m2s
service/openshift-gitops-server-metrics   ClusterIP   172.30.48.223    <none>        8083/TCP            2m2s

NAME                                                         READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/cluster                                      1/1     1            1           2m5s
deployment.apps/kam                                          1/1     1            1           2m5s
deployment.apps/openshift-gitops-applicationset-controller   1/1     1            1           2m3s
deployment.apps/openshift-gitops-dex-server                  1/1     1            1           2m3s
deployment.apps/openshift-gitops-redis                       1/1     1            1           2m3s
deployment.apps/openshift-gitops-repo-server                 1/1     1            1           2m3s
deployment.apps/openshift-gitops-server                      1/1     1            1           2m3s

NAME                                                                    DESIRED   CURRENT   READY   AGE
replicaset.apps/cluster-7c4bdc7756                                      1         1         1       2m5s
replicaset.apps/kam-b49647dc5                                           1         1         1       2m5s
replicaset.apps/openshift-gitops-applicationset-controller-574df4d4d7   1         1         1       2m3s
replicaset.apps/openshift-gitops-dex-server-67779567cd                  1         1         1       2m2s
replicaset.apps/openshift-gitops-dex-server-d644d8645                   0         0         0       2m3s
replicaset.apps/openshift-gitops-redis-7867d74fb4                       1         1         1       2m3s
replicaset.apps/openshift-gitops-repo-server-8568c8b7f                  1         1         1       2m3s
replicaset.apps/openshift-gitops-server-78468747f9                      1         1         1       2m3s

NAME                                                       READY   AGE
statefulset.apps/openshift-gitops-application-controller   1/1     2m3s

NAME                                               HOST/PORT                                                                                             PATH   SERVICES                  PORT    TERMINATION            WILDCARD
route.route.openshift.io/kam                       kam-openshift-gitops.apps.ci-ln-r421z0k-72292.origin-ci-int-gce.dev.rhcloud.com                              kam                       8443    passthrough/None       None
route.route.openshift.io/openshift-gitops-server   openshift-gitops-server-openshift-gitops.apps.ci-ln-r421z0k-72292.origin-ci-int-gce.dev.rhcloud.com          openshift-gitops-server   https   passthrough/Redirect   None

@iam-veeramalla found the cause of the issue which he would describe in detail.

[iam-veeramalla]

With k8s v1.24 serviceaccount does not come up with secrets/tokens by default and for Dex SSO we are using service account as oAuthClient.

https://docs.openshift.com/container-platform/4.7/authentication/using-service-accounts-as-oauth-client.html
This is a breaking change. We need to fix it in the operator.