fix(CNV-29761): ensure that tekton crd exists #567

codingben · 2023-06-07T08:42:19Z

What this PR does / why we need it:

Checks if Tekton CRD is installed during Tekton
operands reconciliation and during cleanup
process when listing deprecated ClusterTasks.

Release note:

Ensure Tekton CRD exists during reconciliation and cleanup process.

codingben · 2023-06-07T10:58:34Z

@ksimon1 Also proposed similar solution in 9784b47 (no PR yet). IMO, request.CrdList.CrdExists(tektonCrd) will always return false, since we're not watching Tekton CRD.

codingben · 2023-06-07T10:59:41Z

I'm trying to solve an error that occurred here [1] in a job of [2] PR:

"error":"no matches for kind \"ClusterTask\" in version \"tekton.dev/v1beta1\""

[1] https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/origin-ci-test/pr-logs/pull/kubevirt_hyperconverged-cluster-operator/2349/pull-ci-kubevirt-hyperconverged-cluster-operator-main-hco-e2e-upgrade-prev-index-aws/1665732018995990528/artifacts/hco-e2e-upgrade-prev-index-aws/e2e-upgrade-test/artifacts/kubevirt-must-gather/quay-io-kubevirt-must-gather-sha256-28fe925c8305c0acda782029f983d241d688366dea7a9c83321db8d210402a3b/namespaces/kubevirt-hyperconverged/pods/ssp-operator-6bc677c64c-6z5r8/manager/manager/logs/current.log
[2] kubevirt/hyperconverged-cluster-operator#2349

codingben · 2023-06-07T11:43:51Z

This PR is related to #566 as the error appears in the same log. From HCO CI:

2023-06-05T15:54:49.852185058Z {"level":"error","ts":"2023-06-05T15:54:49Z","logger":"controller-runtime.source","msg":"if kind is a CRD, it should be installed before calling Start","kind":"VirtualMachine.kubevirt.io","error":"no matches for kind \"VirtualMachine\" in version \"kubevirt.io/v1\"","stacktrace":"sigs.k8s.io/controller-runtime/pkg/source.(*Kind).Start.func1.1\n\t/workspace/vendor/sigs.k8s.io/controller-runtime/pkg/source/source.go:143\nk8s.io/apimachinery/pkg/util/wait.runConditionWithCrashProtectionWithContext\n\t/workspace/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:235\nk8s.io/apimachinery/pkg/util/wait.WaitForWithContext\n\t/workspace/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:662\nk8s.io/apimachinery/pkg/util/wait.poll\n\t/workspace/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:596\nk8s.io/apimachinery/pkg/util/wait.PollImmediateUntilWithContext\n\t/workspace/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:547\nsigs.k8s.io/controller-runtime/pkg/source.(*Kind).Start.func1\n\t/workspace/vendor/sigs.k8s.io/controller-runtime/pkg/source/source.go:136"}
2023-06-05T15:54:54.101717335Z {"level":"error","ts":"2023-06-05T15:54:54Z","msg":"Reconciler error","controller":"ssp","controllerGroup":"ssp.kubevirt.io","controllerKind":"SSP","SSP":{"name":"ssp-kubevirt-hyperconverged","namespace":"kubevirt-hyperconverged"},"namespace":"kubevirt-hyperconverged","name":"ssp-kubevirt-hyperconverged","reconcileID":"88d7bebc-81f4-44d1-b833-6cef9117cdad","error":"no matches for kind \"ClusterTask\" in version \"tekton.dev/v1beta1\"","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/workspace/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:329\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/workspace/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:274\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/workspace/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:235"}

internal/operands/tekton-tasks/tekton-tasks.go

ksimon1 · 2023-06-08T08:28:55Z

/lgtm

codingben · 2023-06-08T10:06:45Z

/test e2e-single-node-functests

kubevirt-bot · 2023-06-08T10:06:47Z

@codingben: No presubmit jobs available for kubevirt/ssp-operator@master

In response to this:

/test e2e-single-node-functests

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

codingben · 2023-06-08T10:27:20Z

We can also consider #570 as an alternative solution to that potential issue, since listDeprecatedClusterTasks() was moved from Cleanup() to Reconcile().

codingben · 2023-06-08T14:28:31Z

/test win11-pipeline-example-test

kubevirt-bot · 2023-06-08T14:28:33Z

@codingben: No presubmit jobs available for kubevirt/ssp-operator@master

In response to this:

/test win11-pipeline-example-test

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

codingben · 2023-06-11T07:53:14Z

/test win11-pipeline-example-test

kubevirt-bot · 2023-06-11T07:53:16Z

@codingben: No presubmit jobs available for kubevirt/ssp-operator@master

In response to this:

/test win11-pipeline-example-test

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

codingben · 2023-06-12T08:00:05Z

Yesterday I've reproduced that error in CNV QE cluster, it happened during CNV product removal process. After I've installed Tekton CRD, it's no longer complained about an error, so I'm pretty sure this is caused by defined ClusterTask in Cleanup().

akrejcir · 2023-06-12T08:18:37Z

internal/operands/tekton-tasks/tekton-tasks.go

+	// Solution to optional Tekton CRD is not implemented yet.
+	// Until then, do not check if Tekton CRD exists.
+	//
+	// if !request.CrdList.CrdExists(tektonCrd) {


What is the reason why not use this check instead of checking if feature gate is enabled? Does it not work?

ClusterTasks should be removed on Cleanup() even if the feature gate is disabled, @ksimon1 is that right?
When Tekton CRD does not exist then there are no ClusterTasks to cleanup.

Yes, if crd does not exists, then there are no tasks / clusterTasks/ pipelines

What is the reason why not use this check instead of checking if feature gate is enabled? Does it not work?

We're not watching Tekton CRD, because then it'll throw an error because it thinks it's required CRD and not an optional. Please see #561.

ClusterTasks should be removed on Cleanup() even if the feature gate is disabled, @ksimon1 is that right?

Yes, but then Tekton CRD is required. I assume that if the customer is using TTO, customer likely will have Tekton CRD already installed so the Feature Gate in ssp-operator will be enabled as well.

When Tekton CRD does not exist then there are no ClusterTasks to cleanup.

Yes, but we'll need to have !request.CrdList.CrdExists(tektonCrd) for it, which doesn't work now because we don't have a solution to optional CRDs.

To clarify, CrdList interface watches all CustomResourceDefinition objects in the cluster. The method CrdList.CrdExists() can be used to check if any CRD exists in the cluster.
In this case, we can use it to check if Tekton CRD is present, and only delete ClusterTasks if it is.

We're not watching Tekton CRD, because then it'll throw an error because it thinks it's required CRD and not an optional. Please see #561.

We are not watching objects defined in the Tekton CRD. But we are watching the CRD itself.

Yes, you're absolutely right. Let's use CrdList interface then.

akrejcir · 2023-06-12T08:30:54Z

internal/operands/tekton-tasks/tekton-tasks.go

@@ -215,7 +227,7 @@ func (t *tektonTasks) Cleanup(request *common.Request) ([]common.CleanupResult,
 func listDeprecatedClusterTasks(request *common.Request) ([]pipeline.ClusterTask, error) {
 	deprecatedClusterTasks := &pipeline.ClusterTaskList{}
 	err := request.Client.List(request.Context, deprecatedClusterTasks, &client.MatchingLabels{
-		common.AppKubernetesManagedByLabel: common.AppKubernetesManagedByValue,
+		common.AppKubernetesManagedByLabel: common.TektonAppKubernetesManagedByValue,


If it makes sense to check for both values, please use In operation on the label selector:

labelRequirement, err := labels.NewRequirement(common.AppKubernetesManagedByLabel, selection.In, []string{common.TektonAppKubernetesManagedByValue, common.AppKubernetesManagedByValue}) if err != nil { panic(err) } err = request.Client.List(request.Context, deprecatedClusterTasks, &client.MatchingLabelsSelector{ Selector: labels.NewSelector().Add(*labelRequirement), })

Nice piece of code! Thanks :) ssp-operator will not deploy ClusterTasks (just Tasks) as it was deprecated, only TTO was deploying ClusterTasks, so it makes sense to just use common.TektonAppKubernetesManagedByValue.

ksimon1 · 2023-06-12T09:51:48Z

/lgtm

akrejcir · 2023-06-12T10:10:34Z

In the Cleanup() methods for both operands. I think it will fail when trying to cleanup Tasks and Pipelines. You also need to add them to the if request.CrdList.CrdExists(tektonCrd) {} block.

Checks if Tekton CRD is installed during Tekton operands reconciliation and during cleanup process when listing deprecated ClusterTasks. Signed-off-by: Ben Oukhanov <boukhanov@redhat.com>

sonarcloud · 2023-06-12T10:33:03Z

Kudos, SonarCloud Quality Gate passed!

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
0.0% Duplication

codingben · 2023-06-12T10:33:49Z

@akrejcir Yep, better to be safe than to be worried. Done.

akrejcir · 2023-06-12T10:34:06Z

Thanks.
/lgtm

akrejcir · 2023-06-12T10:36:37Z

/approve

kubevirt-bot · 2023-06-12T10:36:41Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: akrejcir

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [akrejcir]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

codingben · 2023-06-12T13:23:43Z

/test win2k22-pipeline-example-test

kubevirt-bot · 2023-06-12T13:23:45Z

@codingben: No presubmit jobs available for kubevirt/ssp-operator@master

In response to this:

/test win2k22-pipeline-example-test

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

kubevirt-bot added the dco-signoff: yes Indicates the PR's author has DCO signed all their commits. label Jun 7, 2023

kubevirt-bot requested review from 0xFelix and akrejcir June 7, 2023 08:42

ksimon1 reviewed Jun 8, 2023

View reviewed changes

internal/operands/tekton-tasks/tekton-tasks.go Outdated Show resolved Hide resolved

codingben requested a review from ksimon1 June 8, 2023 08:08

kubevirt-bot assigned ksimon1 Jun 8, 2023

kubevirt-bot added the lgtm Indicates that a PR is ready to be merged. label Jun 8, 2023

codingben mentioned this pull request Jun 8, 2023

feat(CNV-29232): remove resources managed by TTO #570

Closed

akrejcir reviewed Jun 12, 2023

View reviewed changes

codingben marked this pull request as draft June 12, 2023 09:36

kubevirt-bot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. and removed lgtm Indicates that a PR is ready to be merged. labels Jun 12, 2023

codingben changed the title ~~fix(CNV-29436): tekton enabled during cleanup~~ fix(CNV-29436): ensure that tekton crd exists Jun 12, 2023

codingben marked this pull request as ready for review June 12, 2023 09:46

kubevirt-bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jun 12, 2023

kubevirt-bot requested a review from lyarwood June 12, 2023 09:46

codingben requested a review from akrejcir June 12, 2023 09:46

kubevirt-bot added lgtm Indicates that a PR is ready to be merged. and removed lgtm Indicates that a PR is ready to be merged. labels Jun 12, 2023

codingben changed the title ~~fix(CNV-29436): ensure that tekton crd exists~~ fix: ensure that tekton crd exists Jun 12, 2023

fix(CNV-29761): ensure that tekton crd exists

d28f6bd

Checks if Tekton CRD is installed during Tekton operands reconciliation and during cleanup process when listing deprecated ClusterTasks. Signed-off-by: Ben Oukhanov <boukhanov@redhat.com>

codingben changed the title ~~fix: ensure that tekton crd exists~~ fix(CNV-29761): ensure that tekton crd exists Jun 12, 2023

kubevirt-bot assigned akrejcir Jun 12, 2023

kubevirt-bot added the lgtm Indicates that a PR is ready to be merged. label Jun 12, 2023

kubevirt-bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jun 12, 2023

kubevirt-bot merged commit 42be785 into kubevirt:master Jun 12, 2023
13 checks passed

codingben deleted the CNV-29436 branch June 28, 2023 14:22

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix(CNV-29761): ensure that tekton crd exists #567

fix(CNV-29761): ensure that tekton crd exists #567

codingben commented Jun 7, 2023 •

edited

codingben commented Jun 7, 2023

codingben commented Jun 7, 2023

codingben commented Jun 7, 2023 •

edited

ksimon1 commented Jun 8, 2023

codingben commented Jun 8, 2023

kubevirt-bot commented Jun 8, 2023

codingben commented Jun 8, 2023

codingben commented Jun 8, 2023

kubevirt-bot commented Jun 8, 2023

codingben commented Jun 11, 2023

kubevirt-bot commented Jun 11, 2023

codingben commented Jun 12, 2023

akrejcir Jun 12, 2023

ksimon1 Jun 12, 2023

codingben Jun 12, 2023

akrejcir Jun 12, 2023 •

edited

codingben Jun 12, 2023

akrejcir Jun 12, 2023

codingben Jun 12, 2023

ksimon1 commented Jun 12, 2023

akrejcir commented Jun 12, 2023

sonarcloud bot commented Jun 12, 2023

codingben commented Jun 12, 2023 •

edited

akrejcir commented Jun 12, 2023

akrejcir commented Jun 12, 2023

kubevirt-bot commented Jun 12, 2023

codingben commented Jun 12, 2023

kubevirt-bot commented Jun 12, 2023

fix(CNV-29761): ensure that tekton crd exists #567

fix(CNV-29761): ensure that tekton crd exists #567

Conversation

codingben commented Jun 7, 2023 • edited

codingben commented Jun 7, 2023

codingben commented Jun 7, 2023

codingben commented Jun 7, 2023 • edited

ksimon1 commented Jun 8, 2023

codingben commented Jun 8, 2023

kubevirt-bot commented Jun 8, 2023

codingben commented Jun 8, 2023

codingben commented Jun 8, 2023

kubevirt-bot commented Jun 8, 2023

codingben commented Jun 11, 2023

kubevirt-bot commented Jun 11, 2023

codingben commented Jun 12, 2023

akrejcir Jun 12, 2023

Choose a reason for hiding this comment

ksimon1 Jun 12, 2023

Choose a reason for hiding this comment

codingben Jun 12, 2023

Choose a reason for hiding this comment

akrejcir Jun 12, 2023 • edited

Choose a reason for hiding this comment

codingben Jun 12, 2023

Choose a reason for hiding this comment

akrejcir Jun 12, 2023

Choose a reason for hiding this comment

codingben Jun 12, 2023

Choose a reason for hiding this comment

ksimon1 commented Jun 12, 2023

akrejcir commented Jun 12, 2023

sonarcloud bot commented Jun 12, 2023

codingben commented Jun 12, 2023 • edited

akrejcir commented Jun 12, 2023

akrejcir commented Jun 12, 2023

kubevirt-bot commented Jun 12, 2023

codingben commented Jun 12, 2023

kubevirt-bot commented Jun 12, 2023

codingben commented Jun 7, 2023 •

edited

codingben commented Jun 7, 2023 •

edited

akrejcir Jun 12, 2023 •

edited

codingben commented Jun 12, 2023 •

edited