Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clean install issue #7998

Closed
Skreet2k opened this issue Nov 16, 2021 · 15 comments
Closed

Clean install issue #7998

Skreet2k opened this issue Nov 16, 2021 · 15 comments
Assignees

Comments

@Skreet2k
Copy link

Skreet2k commented Nov 16, 2021

Hello, i use clean instance Kubernetes v1.22 with jx v3.2.213
With this guide
https://jenkins-x.io/v3/admin/platforms/on-premises/
i have error on add git operator stage:

2021-11-16T20:41:27.561Z | wrote /tmp/generate/tekton-pipelines/tekton-pipeline/tekton-pipeline/templates/tekton-pipelines-controller-deploy.yaml
2021-11-16T20:41:27.561Z | wrote /tmp/generate/tekton-pipelines/tekton-pipeline/tekton-pipeline/templates/tekton-pipelines-webhook-deploy.yaml
2021-11-16T20:41:27.561Z | wrote /tmp/generate/tekton-pipelines/tekton-pipeline/tekton-pipeline/templates/tekton-pipelines-webhook-horizontalpodautoscaler.yaml
2021-11-16T20:41:27.561Z | wrote /tmp/generate/tekton-pipelines/tekton-pipeline/tekton-pipeline/templates/webhook.pipeline.tekton.dev-mutwebhookcfg.yaml
2021-11-16T20:41:27.561Z | wrote /tmp/generate/tekton-pipelines/tekton-pipeline/tekton-pipeline/templates/config.webhook.pipeline.tekton.dev-valwebhookcfg.yaml
2021-11-16T20:41:27.561Z | wrote /tmp/generate/tekton-pipelines/tekton-pipeline/tekton-pipeline/templates/validation.webhook.pipeline.tekton.dev-valwebhookcfg.yaml
2021-11-16T20:41:27.561Z |
2021-11-16T20:41:27.561Z |
2021-11-16T20:41:27.562Z | jx gitops split --dir /tmp/generate
2021-11-16T20:41:27.934Z | jx gitops rename --dir /tmp/generate
2021-11-16T20:41:28.352Z | jx gitops helmfile move --output-dir config-root --dir /tmp/generate --dir-includes-release-name
2021-11-16T20:41:28.902Z | jx secret convert --source-dir config-root -r jx-vault
2021-11-16T20:41:29.225Z | jx secret replicate --selector secret.jenkins-x.io/replica-source=true
2021-11-16T20:41:29.656Z | jx gitops namespace --dir-mode --dir config-root/namespaces
2021-11-16T20:41:30.067Z | jx gitops helmfile report
2021-11-16T20:41:30.538Z | namespace jx
2021-11-16T20:41:32.095Z | ignoring Ingress in file config-root/namespaces/jx/jxboot-helmfile-resources/bucketrepo-ingress.yaml with api version networking.k8s.io/v1
2021-11-16T20:41:32.097Z | ignoring Ingress in file config-root/namespaces/jx/jxboot-helmfile-resources/hook-ingress.yaml with api version networking.k8s.io/v1
2021-11-16T20:41:32.100Z | found jxboot-helmfile-resources version: 1.1.0 https://github.com/jenkins-x-charts/jxboot-helmfile-resources
2021-11-16T20:41:32.100Z | WARNING: ignoring release jxgh/jenkins-x-crds in namespace jx as we cannot find any generated resources but there is no conditional
2021-11-16T20:41:33.631Z | ignoring Ingress in file config-root/namespaces/jx/jx-pipelines-visualizer/jx-pipelines-visualizer-ingress.yaml with api version networking.k8s.io/v1
2021-11-16T20:41:33.632Z | found jx-pipelines-visualizer version: 1.7.5 https://github.com/jenkins-x/jx-pipelines-visualizer
2021-11-16T20:41:35.131Z | found jx-preview version: 0.0.192 https://github.com/jenkins-x-plugins/jx-preview
2021-11-16T20:41:36.677Z | found lighthouse version: 1.1.52 https://github.com/jenkins-x/lighthouse
2021-11-16T20:41:38.235Z | found bucketrepo version: 0.1.67
2021-11-16T20:41:39.779Z | found jx-build-controller version: 0.3.9 https://jenkins-x.io/
2021-11-16T20:41:39.779Z | WARNING: ignoring release jxgh/local-external-secrets in namespace jx as we cannot find any generated resources but there is no conditional
2021-11-16T20:41:41.324Z | found jx-kh-check version: 0.0.78 https://jenkins-x.io/
2021-11-16T20:41:41.324Z |
2021-11-16T20:41:41.325Z | namespace jx
2021-11-16T20:41:42.922Z | ignoring Ingress in file config-root/namespaces/jx/jxboot-helmfile-resources/bucketrepo-ingress.yaml with api version networking.k8s.io/v1
2021-11-16T20:41:42.924Z | ignoring Ingress in file config-root/namespaces/jx/jxboot-helmfile-resources/hook-ingress.yaml with api version networking.k8s.io/v1
2021-11-16T20:41:42.927Z | found jxboot-helmfile-resources version: 1.1.0 https://github.com/jenkins-x-charts/jxboot-helmfile-resources
2021-11-16T20:41:42.927Z | WARNING: ignoring release jxgh/jenkins-x-crds in namespace jx as we cannot find any generated resources but there is no conditional
2021-11-16T20:41:44.421Z | ignoring Ingress in file config-root/namespaces/jx/jx-pipelines-visualizer/jx-pipelines-visualizer-ingress.yaml with api version networking.k8s.io/v1
2021-11-16T20:41:44.421Z | found jx-pipelines-visualizer version: 1.7.5 https://github.com/jenkins-x/jx-pipelines-visualizer
2021-11-16T20:41:45.978Z | found jx-preview version: 0.0.192 https://github.com/jenkins-x-plugins/jx-preview
2021-11-16T20:41:47.482Z | found lighthouse version: 1.1.52 https://github.com/jenkins-x/lighthouse
2021-11-16T20:41:49.026Z | found bucketrepo version: 0.1.67
2021-11-16T20:41:50.640Z | found jx-build-controller version: 0.3.9 https://jenkins-x.io/
2021-11-16T20:41:50.640Z | WARNING: ignoring release jxgh/local-external-secrets in namespace jx as we cannot find any generated resources but there is no conditional
2021-11-16T20:41:52.122Z | found jx-kh-check version: 0.0.78 https://jenkins-x.io/
2021-11-16T20:41:52.122Z |
2021-11-16T20:41:52.122Z | namespace kuberhealthy
2021-11-16T20:41:52.634Z | found kuberhealthy version: 64 https://comcast.github.io/kuberhealthy/
2021-11-16T20:41:54.161Z | found jx-kh-check version: 0.0.78 https://jenkins-x.io/
2021-11-16T20:41:54.161Z |
2021-11-16T20:41:54.161Z | namespace kuberhealthy
2021-11-16T20:41:54.658Z | found kuberhealthy version: 64 https://comcast.github.io/kuberhealthy/
2021-11-16T20:41:56.168Z | found jx-kh-check version: 0.0.78 https://jenkins-x.io/
2021-11-16T20:41:56.168Z |
2021-11-16T20:41:56.168Z | namespace nginx
2021-11-16T20:41:58.271Z | found ingress-nginx version: 3.12.0 https://github.com/kubernetes/ingress-nginx
2021-11-16T20:41:58.271Z |
2021-11-16T20:41:58.271Z | namespace nginx
2021-11-16T20:42:00.192Z | found ingress-nginx version: 3.12.0 https://github.com/kubernetes/ingress-nginx
2021-11-16T20:42:00.192Z |
2021-11-16T20:42:00.192Z | namespace tekton-pipelines
2021-11-16T20:42:01.668Z | found tekton-pipeline version: 0.27.2 https://github.com/cdfoundation/tekton-helm-chart
2021-11-16T20:42:01.668Z |
2021-11-16T20:42:01.668Z | namespace tekton-pipelines
2021-11-16T20:42:03.056Z | found tekton-pipeline version: 0.27.2 https://github.com/cdfoundation/tekton-helm-chart
2021-11-16T20:42:03.056Z |
2021-11-16T20:42:03.058Z | saved docs/releases.yaml
2021-11-16T20:42:03.058Z | saved docs/README.md
2021-11-16T20:42:03.060Z | jx gitops scheduler
2021-11-16T20:42:03.492Z | jx gitops hash --pod-spec --kind Deployment -s config-root/namespaces/jx/lighthouse-config/config-cm.yaml -s config-root/namespaces/jx/lighthouse-config/plugins-cm.yaml -d config-root/namespaces/jx/lighthouse
2021-11-16T20:42:03.854Z | jx gitops label --dir config-root/cluster gitops.jenkins-x.io/pipeline=cluster
2021-11-16T20:42:04.274Z | jx gitops label --dir config-root/customresourcedefinitions gitops.jenkins-x.io/pipeline=customresourcedefinitions
2021-11-16T20:42:04.574Z | jx gitops label --dir config-root/namespaces gitops.jenkins-x.io/pipeline=namespaces
2021-11-16T20:42:04.858Z | jx gitops annotate --dir config-root --selector app=pusher-wave kapp.k14s.io/change-group=apps.jenkins-x.io/pusher-wave
2021-11-16T20:42:05.307Z | jx gitops annotate --dir config-root --selector app.kubernetes.io/name=ingress-nginx kapp.k14s.io/change-group=apps.jenkins-x.io/ingress-nginx
2021-11-16T20:42:05.624Z | jx gitops label --dir config-root/cluster --kind=Namespace team=jx
2021-11-16T20:42:05.919Z | jx gitops annotate --dir config-root/namespaces --kind Deployment --selector app=pusher-wave --invert-selector wave.pusher.com/update-on-config-change=true
2021-11-16T20:42:06.215Z | using kubectl to apply resources
2021-11-16T20:42:06.215Z | kubectl apply --force --prune -l=gitops.jenkins-x.io/pipeline=customresourcedefinitions -R -f config-root/customresourcedefinitions
2021-11-16T20:42:06.627Z | customresourcedefinition.apiextensions.k8s.io/previews.preview.jenkins.io unchanged
2021-11-16T20:42:06.669Z | customresourcedefinition.apiextensions.k8s.io/lighthousebreakpoints.lighthouse.jenkins.io unchanged
2021-11-16T20:42:06.718Z | customresourcedefinition.apiextensions.k8s.io/lighthousejobs.lighthouse.jenkins.io unchanged
2021-11-16T20:42:06.771Z | customresourcedefinition.apiextensions.k8s.io/externalsecrets.kubernetes-client.io unchanged
2021-11-16T20:42:07.098Z | customresourcedefinition.apiextensions.k8s.io/clustertasks.tekton.dev configured
2021-11-16T20:42:07.138Z | customresourcedefinition.apiextensions.k8s.io/conditions.tekton.dev unchanged
2021-11-16T20:42:07.181Z | customresourcedefinition.apiextensions.k8s.io/pipelineresources.tekton.dev unchanged
2021-11-16T20:42:07.277Z | customresourcedefinition.apiextensions.k8s.io/pipelineruns.tekton.dev configured
2021-11-16T20:42:07.367Z | customresourcedefinition.apiextensions.k8s.io/pipelines.tekton.dev configured
2021-11-16T20:42:07.463Z | customresourcedefinition.apiextensions.k8s.io/runs.tekton.dev configured
2021-11-16T20:42:07.557Z | customresourcedefinition.apiextensions.k8s.io/taskruns.tekton.dev configured
2021-11-16T20:42:07.645Z | customresourcedefinition.apiextensions.k8s.io/tasks.tekton.dev configured
2021-11-16T20:42:07.645Z | error validating "config-root/customresourcedefinitions/jx/jenkins-x-crds/environments.jenkins.io-crd.yaml": error validating data: [ValidationError(CustomResourceDefinition.spec): unknown field "additionalPrinterColumns" in io.k8s.apiextensions-apiserver.pkg.apis.apiextensions.v1.CustomResourceDefinitionSpec, ValidationError(CustomResourceDefinition.spec): unknown field "validation" in io.k8s.apiextensions-apiserver.pkg.apis.apiextensions.v1.CustomResourceDefinitionSpec, ValidationError(CustomResourceDefinition.spec): unknown field "version" in io.k8s.apiextensions-apiserver.pkg.apis.apiextensions.v1.CustomResourceDefinitionSpec]; if you choose to ignore these errors, turn validation off with --validate=false
2021-11-16T20:42:07.645Z | error validating "config-root/customresourcedefinitions/jx/jenkins-x-crds/pipelineactivities.jenkins.io-crd.yaml": error validating data: [ValidationError(CustomResourceDefinition.spec): unknown field "additionalPrinterColumns" in io.k8s.apiextensions-apiserver.pkg.apis.apiextensions.v1.CustomResourceDefinitionSpec, ValidationError(CustomResourceDefinition.spec): unknown field "validation" in io.k8s.apiextensions-apiserver.pkg.apis.apiextensions.v1.CustomResourceDefinitionSpec, ValidationError(CustomResourceDefinition.spec): unknown field "version" in io.k8s.apiextensions-apiserver.pkg.apis.apiextensions.v1.CustomResourceDefinitionSpec]; if you choose to ignore these errors, turn validation off with --validate=false
2021-11-16T20:42:07.645Z | error validating "config-root/customresourcedefinitions/jx/jenkins-x-crds/releases.jenkins.io-crd.yaml": error validating data: [ValidationError(CustomResourceDefinition.spec): unknown field "additionalPrinterColumns" in io.k8s.apiextensions-apiserver.pkg.apis.apiextensions.v1.CustomResourceDefinitionSpec, ValidationError(CustomResourceDefinition.spec): unknown field "validation" in io.k8s.apiextensions-apiserver.pkg.apis.apiextensions.v1.CustomResourceDefinitionSpec, ValidationError(CustomResourceDefinition.spec): unknown field "version" in io.k8s.apiextensions-apiserver.pkg.apis.apiextensions.v1.CustomResourceDefinitionSpec]; if you choose to ignore these errors, turn validation off with --validate=false
2021-11-16T20:42:07.645Z | error validating "config-root/customresourcedefinitions/jx/jenkins-x-crds/sourcerepositories.jenkins.io-crd.yaml": error validating data: [ValidationError(CustomResourceDefinition.spec): unknown field "additionalPrinterColumns" in io.k8s.apiextensions-apiserver.pkg.apis.apiextensions.v1.CustomResourceDefinitionSpec, ValidationError(CustomResourceDefinition.spec): unknown field "validation" in io.k8s.apiextensions-apiserver.pkg.apis.apiextensions.v1.CustomResourceDefinitionSpec, ValidationError(CustomResourceDefinition.spec): unknown field "version" in io.k8s.apiextensions-apiserver.pkg.apis.apiextensions.v1.CustomResourceDefinitionSpec]; if you choose to ignore these errors, turn validation off with --validate=false
2021-11-16T20:42:07.645Z | unable to recognize "config-root/customresourcedefinitions/kuberhealthy/kuberhealthy/khchecks.comcast.github.io-crd.yaml": no matches for kind "CustomResourceDefinition" in version "apiextensions.k8s.io/v1beta1"
2021-11-16T20:42:07.645Z | unable to recognize "config-root/customresourcedefinitions/kuberhealthy/kuberhealthy/khjobs.comcast.github.io-crd.yaml": no matches for kind "CustomResourceDefinition" in version "apiextensions.k8s.io/v1beta1"
2021-11-16T20:42:07.645Z | unable to recognize "config-root/customresourcedefinitions/kuberhealthy/kuberhealthy/khstates.comcast.github.io-crd.yaml": no matches for kind "CustomResourceDefinition" in version "apiextensions.k8s.io/v1beta1"
2021-11-16T20:42:07.653Z | make[1]: Leaving directory '/workspace/source'
2021-11-16T20:42:07.653Z | make[1]: *** [versionStream/src/Makefile.mk:288: kubectl-apply] Error 1
2021-11-16T20:42:07.653Z | error: failed to regenerate: failed to regenerate phase 1: failed to run 'make regen-phase-1' command in directory '.', output: ''
2021-11-16T20:42:07.654Z | make: *** [versionStream/src/Makefile.mk:242: regen-check] Error 1`

in k8s failed started job jx-boot:3.2.213

@ankitm123
Copy link
Member

Most of the test we did were on 1.21, I will spin up a local cluster, and create fixes to make it work with 1.22 this weekend.

@sebinbenjamin
Copy link

I have got the issue replicated in a local cluster. I guess this is related to https://kubernetes.io/blog/2021/07/14/upcoming-changes-in-kubernetes-1-22/ . I am hoping this would be ok if I switch back to 1.21 🤞 .

@ankitm123
Copy link
Member

I am hoping this would be ok if I switch back to 1.21

Yes 🤞

@Skreet2k
Copy link
Author

Do you work on upgrade for resolve problem with 1.22. Or better downgrade to 1.21?

@ankitm123
Copy link
Member

ankitm123 commented Dec 11, 2021

With 1.23 out,it would be good to fix this for 1.22. So for local testing, I use k3s (and in my company I use AWS), and the stable channel still does not have 1.22 (they are on 1.21.7 last time I checked and AWS still does not have 1.22).

I could use the latest channel of k3s which has 1.22.4, but there might be issues with k3s which might make debugging JX issues harder, but I will give it a shot, if stable k3s channel does not have a 1.22 release by Dec 17.

Once I can get a 1.22 cluster on k3s, I will check what needs to be done to fix it.
For now, it's good to stick to 1.21.

@ankitm123 ankitm123 self-assigned this Dec 11, 2021
@faisalthaheem
Copy link

Perhaps update the documentation on k3s (https://jenkins-x.io/v3/admin/platforms/k3s/) to mention it's compatible with only version 1.21 at the moment?

@ankitm123
Copy link
Member

Jenkins X as a whole is not compatible with 1.22 yet, we still use the beta apis for crds.
I think it's a good idea to mention which versions of kubernetes Jenkins X is compatible with, or atleast tested with.

@necromashka
Copy link

Jenkins X as a whole is not compatible with 1.22 yet, we still use the beta apis for crds. I think it's a good idea to mention which versions of kubernetes Jenkins X is compatible with, or atleast tested with.

Hi
Is any progress for this issue at this time?

@ankitm123
Copy link
Member

Yes, I have a PR re-generating crds to work with kubernetes 1.22: jenkins-x/jx-api#162. Once that is done (need to re-generate open api validation schema) and merged, the last issue is upgrading tekton, and seeing if it works.

@ankitm123
Copy link
Member

If anyone wants to try, and give us some feedback, then the manifests are in this PR: jenkins-x-charts/jenkins-x-crds#8 (you can try my fork with branch - support-k8s-1.22 https://github.com/ankitm123/jenkins-x-crds/tree/support-k8s-1.22). Go to the templates folder, and run kubectl apply -f ., let me know if you run into issues.

Is any one running kubernetes 1.21, and wants to try it, that would also be nice! We dont want to break existing clusters ...

@ankitm123
Copy link
Member

#8032 -> Issue tracking kuberntes 1.22 support.

@pk185235
Copy link

do we have an update on when it is going to be compatible with 1.22? i have the same issue deploying the operator and downgrading is not an option at the moment

@ankitm123
Copy link
Member

https://github.com/jenkins-x-charts/jenkins-x-crds/releases/tag/v3.0.12 -> Use version 3.0.12 for crds (it got merged today, but not yet available in the version stream). Change here to 3.0.12: https://github.com/DexaiRobotics/jx3-eks-vault/blob/bbceb058a4a7b5afcf21f3aea1d50d623719cdc5/versionStream/charts/jxgh/jx-build-controller/defaults.yaml#L2

The only other PR that remains is: cdfoundation/tekton-helm-chart#17. which should get merged soon.

@pk185235
Copy link

https://github.com/jenkins-x-charts/jenkins-x-crds/releases/tag/v3.0.12 -> Use version 3.0.12 for crds (it got merged today, but not yet available in the version stream). Change here to 3.0.12: https://github.com/DexaiRobotics/jx3-eks-vault/blob/bbceb058a4a7b5afcf21f3aea1d50d623719cdc5/versionStream/charts/jxgh/jx-build-controller/defaults.yaml#L2

The only other PR that remains is: cdfoundation/tekton-helm-chart#17. which should get merged soon.

Can you please create a more step by step explanation of what is needed to be done in order to deploy jenkinsx in kubernetes 1.22? I cannot follow up on some of the instructions provided

@babadofar
Copy link

Closing this issue as we have other issues covering kubernetes 1.22 upgrade #8032

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

7 participants