CCO-372: Azure Workload Identity info in CredsRequests creates a Secret #587

bentito · 2023-08-02T16:17:58Z

Adds Azure Workload Identity (Timed Access Token) support and e2e test and unit tests.

pkg/azure/actuator_test.go should get a review for changes that were needed to pre-existing unit tests. The changes were needed because of the requirements of the new detection for Workload Identity in-use flow.

Known issues:

- ~~Still need to verify OCPBUGS filed against~~ the AWS STS work, particularly around properly setting status, conditions and lastSync times that were fixed for AWS STS are not brought back here for Azure.
* Reviewed the several bug fix PRs against the AWS STS work, most fixes were in credentialsrequest_controller so their all present for this Azure branch as well, just OCPBUGS-16684 had an actuator change that seemed to be needed here.
- This should be e2e testable with a PR in release which actually calls the new Make target: test-e2e-azident with the e2e-azure-manual-oidc workflow
* PR to add calling make target to azure e2e is here: Add cmd make test-e2e-azident to e2e-azure-manual-oidc release#42126

openshift-ci-robot · 2023-08-02T16:18:01Z

@bentito: This pull request references CCO-372 which is a valid jira issue.

In response to this:

WIP: Adds Azure Workload Identity (Timed Access Token) support and e2e test (and unit tests eventually).

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-ci · 2023-08-02T16:18:01Z

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

openshift-ci-robot · 2023-08-08T06:42:31Z

@bentito: This pull request references CCO-372 which is a valid jira issue.

In response to this:

WIP: Adds Azure Workload Identity (Timed Access Token) support and e2e test and unit tests.

Known issue: Still need to verify OCPBUGS filed against the AWS STS work, particularly around properly setting status, conditions and lastSync times that were fixed for AWS STS are not brought back here for Azure.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-ci-robot · 2023-08-08T06:47:15Z

@bentito: This pull request references CCO-372 which is a valid jira issue.

In response to this:

WIP: Adds Azure Workload Identity (Timed Access Token) support and e2e test and unit tests.

Known issues:

Still need to verify OCPBUGS filed against the AWS STS work, particularly around properly setting status, conditions and lastSync times that were fixed for AWS STS are not brought back here for Azure.

This should be e2e testable with a PR in release which actually calls the new Make target: test-e2e-azident with the e2e-azure-manual-oidc workflow

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-ci-robot · 2023-08-08T06:50:30Z

@bentito: This pull request references CCO-372 which is a valid jira issue.

In response to this:

WIP: Adds Azure Workload Identity (Timed Access Token) support and e2e test and unit tests.

pkg/azure/actuator_test.go should get a review for changes that were needed to pre-existing unit tests. The changes were needed because of the requirements of the new detection for Workload Identity in-use flow.

Known issues:

Still need to verify OCPBUGS filed against the AWS STS work, particularly around properly setting status, conditions and lastSync times that were fixed for AWS STS are not brought back here for Azure.

This should be e2e testable with a PR in release which actually calls the new Make target: test-e2e-azident with the e2e-azure-manual-oidc workflow

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

codecov · 2023-08-08T07:20:34Z

Codecov Report

Merging #587 (7c970c7) into master (5a6da9c) will increase coverage by 0.06%.
Report is 4 commits behind head on master.
The diff coverage is 68.75%.

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #587      +/-   ##
==========================================
+ Coverage   47.90%   47.96%   +0.06%     
==========================================
  Files          96       96              
  Lines       11680    11737      +57     
==========================================
+ Hits         5595     5630      +35     
- Misses       5465     5483      +18     
- Partials      620      624       +4

Files	Coverage Δ
...redentialsrequest/credentialsrequest_controller.go	`48.78% <0.00%> (ø)`
pkg/azure/actuator.go	`60.97% <69.62%> (+7.27%)`	⬆️

... and 3 files with indirect coverage changes

openshift-ci-robot · 2023-08-09T05:44:25Z

@bentito: This pull request references CCO-372 which is a valid jira issue.

In response to this:

WIP: Adds Azure Workload Identity (Timed Access Token) support and e2e test and unit tests.

pkg/azure/actuator_test.go should get a review for changes that were needed to pre-existing unit tests. The changes were needed because of the requirements of the new detection for Workload Identity in-use flow.

Known issues:

- Still need to verify OCPBUGS filed against the AWS STS work, particularly around properly setting status, conditions and lastSync times that were fixed for AWS STS are not brought back here for Azure.

Reviewed the several bug fix PRs against the AWS STS work, most fixes were in credentialsrequest_controller so their all present for this Azure branch as well, just OCPBUGS-16684 had an actuator change that seemed to be needed here.

- This should be e2e testable with a PR in release which actually calls the new Make target: test-e2e-azident with the e2e-azure-manual-oidc workflow

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-ci-robot · 2023-08-09T05:44:28Z

@bentito: This pull request references CCO-372 which is a valid jira issue.

In response to this:

WIP: Adds Azure Workload Identity (Timed Access Token) support and e2e test and unit tests.

pkg/azure/actuator_test.go should get a review for changes that were needed to pre-existing unit tests. The changes were needed because of the requirements of the new detection for Workload Identity in-use flow.

Known issues:

- Still need to verify OCPBUGS filed against the AWS STS work, particularly around properly setting status, conditions and lastSync times that were fixed for AWS STS are not brought back here for Azure.

Reviewed the several bug fix PRs against the AWS STS work, most fixes were in credentialsrequest_controller so their all present for this Azure branch as well, just OCPBUGS-16684 had an actuator change that seemed to be needed here.

- This should be e2e testable with a PR in release which actually calls the new Make target: test-e2e-azident with the e2e-azure-manual-oidc workflow

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

bentito · 2023-08-09T06:13:53Z

/retest e2e-hypershift

openshift-ci · 2023-08-09T06:13:57Z

@bentito: The /retest command does not accept any targets.
The following commands are available to trigger required jobs:

/test coverage
/test e2e-aws-ovn
/test e2e-hypershift
/test e2e-upgrade
/test images
/test unit
/test verify
/test verify-deps

The following commands are available to trigger optional jobs:

/test e2e-aws-manual-oidc
/test e2e-azure
/test e2e-azure-manual-oidc
/test e2e-azure-upgrade
/test e2e-gcp
/test e2e-gcp-manual-oidc
/test e2e-openstack
/test e2e-openstack-parallel

Use /test all to run the following jobs that were automatically triggered:

pull-ci-openshift-cloud-credential-operator-master-coverage
pull-ci-openshift-cloud-credential-operator-master-e2e-aws-ovn
pull-ci-openshift-cloud-credential-operator-master-e2e-hypershift
pull-ci-openshift-cloud-credential-operator-master-e2e-upgrade
pull-ci-openshift-cloud-credential-operator-master-images
pull-ci-openshift-cloud-credential-operator-master-unit
pull-ci-openshift-cloud-credential-operator-master-verify
pull-ci-openshift-cloud-credential-operator-master-verify-deps

In response to this:

/retest e2e-hypershift

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

bentito · 2023-08-09T06:14:29Z

/test e2e-hypershift

openshift-ci-robot · 2023-08-09T06:18:35Z

@bentito: This pull request references CCO-372 which is a valid jira issue.

In response to this:

WIP: Adds Azure Workload Identity (Timed Access Token) support and e2e test and unit tests.

pkg/azure/actuator_test.go should get a review for changes that were needed to pre-existing unit tests. The changes were needed because of the requirements of the new detection for Workload Identity in-use flow.

Known issues:

- Still need to verify OCPBUGS filed against the AWS STS work, particularly around properly setting status, conditions and lastSync times that were fixed for AWS STS are not brought back here for Azure.

Reviewed the several bug fix PRs against the AWS STS work, most fixes were in credentialsrequest_controller so their all present for this Azure branch as well, just OCPBUGS-16684 had an actuator change that seemed to be needed here.

- This should be e2e testable with a PR in release which actually calls the new Make target: test-e2e-azident with the e2e-azure-manual-oidc workflow

PR to add calling make target to azure e2e is here: Add cmd make test-e2e-azident to e2e-azure-manual-oidc release#42126

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-ci-robot · 2023-08-09T06:18:39Z

@bentito: This pull request references CCO-372 which is a valid jira issue.

In response to this:

WIP: Adds Azure Workload Identity (Timed Access Token) support and e2e test and unit tests.

pkg/azure/actuator_test.go should get a review for changes that were needed to pre-existing unit tests. The changes were needed because of the requirements of the new detection for Workload Identity in-use flow.

Known issues:

- Still need to verify OCPBUGS filed against the AWS STS work, particularly around properly setting status, conditions and lastSync times that were fixed for AWS STS are not brought back here for Azure.

Reviewed the several bug fix PRs against the AWS STS work, most fixes were in credentialsrequest_controller so their all present for this Azure branch as well, just OCPBUGS-16684 had an actuator change that seemed to be needed here.

- This should be e2e testable with a PR in release which actually calls the new Make target: test-e2e-azident with the e2e-azure-manual-oidc workflow

PR to add calling make target to azure e2e is here: Add cmd make test-e2e-azident to e2e-azure-manual-oidc release#42126

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-ci-robot · 2023-08-09T06:19:08Z

@bentito: This pull request references CCO-372 which is a valid jira issue.

In response to this:

WIP: Adds Azure Workload Identity (Timed Access Token) support and e2e test and unit tests.

pkg/azure/actuator_test.go should get a review for changes that were needed to pre-existing unit tests. The changes were needed because of the requirements of the new detection for Workload Identity in-use flow.

Known issues:

- ~~Still need to verify OCPBUGS filed against~~ the AWS STS work, particularly around properly setting status, conditions and lastSync times that were fixed for AWS STS are not brought back here for Azure.

Reviewed the several bug fix PRs against the AWS STS work, most fixes were in credentialsrequest_controller so their all present for this Azure branch as well, just OCPBUGS-16684 had an actuator change that seemed to be needed here.

- This should be e2e testable with a PR in release which actually calls the new Make target: test-e2e-azident with the e2e-azure-manual-oidc workflow

PR to add calling make target to azure e2e is here: Add cmd make test-e2e-azident to e2e-azure-manual-oidc release#42126

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

bentito · 2023-08-09T13:35:46Z

/test e2e-hypershift

bentito · 2023-08-09T14:58:47Z

/test e2e-hypershift

bentito · 2023-08-09T15:08:46Z

/test e2e-azure-manual-oidc

openshift-ci-robot · 2023-08-09T15:12:45Z

@bentito: This pull request references CCO-372 which is a valid jira issue.

In response to this:

Adds Azure Workload Identity (Timed Access Token) support and e2e test and unit tests.

pkg/azure/actuator_test.go should get a review for changes that were needed to pre-existing unit tests. The changes were needed because of the requirements of the new detection for Workload Identity in-use flow.

Known issues:

- ~~Still need to verify OCPBUGS filed against~~ the AWS STS work, particularly around properly setting status, conditions and lastSync times that were fixed for AWS STS are not brought back here for Azure.

Reviewed the several bug fix PRs against the AWS STS work, most fixes were in credentialsrequest_controller so their all present for this Azure branch as well, just OCPBUGS-16684 had an actuator change that seemed to be needed here.

- This should be e2e testable with a PR in release which actually calls the new Make target: test-e2e-azident with the e2e-azure-manual-oidc workflow

PR to add calling make target to azure e2e is here: Add cmd make test-e2e-azident to e2e-azure-manual-oidc release#42126

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

bentito · 2023-08-10T05:11:01Z

/test e2e-azure-manual-oidc

bentito · 2023-08-10T15:31:32Z

@abutcher we're running into trouble here with Azure that we didn't see on AWS:

cloud-credential-operator/pkg/operator/credentialsrequest/credentialsrequest_controller.go

Line 683 in 973abf9

credsExists, err := r.Actuator.Exists(ctx, cr)

With the error at runtime looking like:

time="2023-08-05T02:35:39Z" level=error msg="error checking whether credentials already exists: Secret \"azure-credentials\" not found" controller=credreq cr=openshift-cloud-credential-operator/nginx secret=default/nginx-credentials

There is certainly more logic in the non-STSDetected flow after the else before the .Exists() check, but I'm not sure what in it might need to be pulled up to the STS flow section? Any ideas?

gallettilance · 2023-11-28T16:31:55Z

/test e2e-upgrade

gallettilance · 2023-11-28T16:32:07Z

/test e2e-aws-ovn

gallettilance · 2023-11-28T18:14:30Z

cred-req-secret-creationdeletion.webm

@jstuever here is a demo of the secret being created and deleted

jstuever · 2023-11-28T18:22:55Z

/test e2e-azure e2e-azure-upgrade

gallettilance · 2023-11-28T23:15:24Z

cred-req-workload-identity.webm

Update based on our convo @jstuever

no errors for "old" manual mode
updates of the secret to make it look exactly as the ccoctl tool does

gallettilance

Some notes for reviewers

gallettilance · 2023-11-29T16:20:59Z

pkg/azure/actuator.go

 		return err
 	}
+	if credentialsMode == operatorv1.CloudCredentialsModeManual {
+		logger.Debug("running delete in manual mode")
+		return nil


Note to reviewers: the delete works because of the annotation on the secret

gallettilance · 2023-11-29T16:22:09Z

pkg/azure/actuator.go

+		return true, err
+	}
+	if credentialsMode == operatorv1.CloudCredentialsModeManual {
+		return true, nil


Note to reviewers: what's in the credReq should be the source of truth propagated down to the secret.

gallettilance · 2023-11-29T16:23:49Z

pkg/azure/actuator.go

+		}
+		// Check for old Manual Mode where all 4 fields are empty - defaulting to old behavior
+		// where CCO exists and the secret is created manually
+		if azureProviderSpec.AzureClientID == "" && azureProviderSpec.AzureTenantID == "" && azureProviderSpec.AzureSubscriptionID == "" && azureProviderSpec.AzureRegion == "" {


Note to reviewers: if none of the fields are provided, we assume the creator of the credReq is intending to use the old manual mode where the secrets are created by the ccoctl tool and CCO doesn't do anything (as is the case with openshift operators today).

gallettilance · 2023-11-29T16:24:57Z

pkg/azure/actuator.go

+		if azureProviderSpec.AzureClientID == "" && azureProviderSpec.AzureTenantID == "" && azureProviderSpec.AzureSubscriptionID == "" && azureProviderSpec.AzureRegion == "" {
+			return nil
+		}
+		err = validateAzureProviderSpec(*azureProviderSpec)


Note to reviewers: if some but not all fields required for this new manual mode workload identity workflow are set then we should error

gallettilance · 2023-11-29T16:27:21Z

pkg/azure/actuator.go

@@ -391,16 +460,24 @@ func (a *Actuator) syncCredentialSecrets(ctx context.Context, cr *minterv1.Crede
 			secret.Annotations = map[string]string{}
 		}
 		secret.Annotations[minterv1.AnnotationCredentialsRequest] = fmt.Sprintf("%s/%s", cr.Namespace, cr.Name)
+		if desiredSecret.Data == nil {


Note to reviewers: this if statement checks that the desiredSecret is from the new manual mode workload identity workflow since it does not set the Data field

gallettilance · 2023-11-29T16:29:06Z

pkg/azure/actuator_test.go

@@ -463,9 +509,31 @@ func TestActuator(t *testing.T) {
 		},
 		{
 			name:        "Mint annotation",
-			expectedErr: fmt.Errorf("error determining whether a credentials update is needed"),
+			expectedErr: fmt.Errorf("unexpected value or missing cloudcredential.openshift.io/mode annotation on admin credentials Secret"),


Note to reviewers: this change is a result of erroring earlier in the process as a result of checking for the cloudcredentials mode.

gallettilance · 2023-11-29T16:29:47Z

pkg/azure/const.go

+	AzureResourcePrefix     = "azure_resource_prefix"
+	AzureSubscriptionID     = "azure_subscription_id"
+	AzureTenantID           = "azure_tenant_id"
+	AzureFederatedTokenFile = "azure_federated_token_file"


Note to reviewers: just adding the AzureFederatedTokenFile

jstuever · 2023-11-29T17:28:44Z

/test e2e-azure e2e-azure-upgrade

jstuever · 2023-11-29T17:29:58Z

pkg/azure/actuator.go

+		return true, nil
+	}
+	if credentialsMode == operatorv1.CloudCredentialsModeMint {
+		return true, errors.New("mint mode is invalid")


Should this return false?

jstuever · 2023-11-29T17:31:13Z

pkg/azure/actuator.go

+		return true, err
+	}
+	if credentialsMode == operatorv1.CloudCredentialsModeManual {
+		return true, nil


Ideally, we would actually check if the secret differs from the source of truth similar to AWS mint mode. However, this can be taken away as tech debt.

jstuever · 2023-11-29T18:19:10Z

/test e2e-azure
/test e2e-azure-upgrade

jstuever · 2023-11-29T19:58:15Z

/test e2e-hypershift

jstuever

/lgtm
/approve

openshift-ci · 2023-11-29T21:34:46Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: bentito, jstuever

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [jstuever]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

openshift-ci · 2023-11-29T21:50:28Z

@bentito: all tests passed!

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

openshift-bot · 2023-11-30T03:43:42Z

[ART PR BUILD NOTIFIER]

This PR has been included in build ose-cloud-credential-operator-container-v4.15.0-202311300249.p0.ge295bb2.assembly.stream for distgit ose-cloud-credential-operator.
All builds following this will include this PR.

gallettilance · 2023-12-18T13:19:20Z

/cherry-pick release-4.14

openshift-cherrypick-robot · 2023-12-18T13:20:13Z

@gallettilance: new pull request created: #643

In response to this:

/cherry-pick release-4.14

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Aug 2, 2023

openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Aug 2, 2023

bentito marked this pull request as ready for review August 8, 2023 06:42

openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Aug 8, 2023

openshift-ci bot requested review from abutcher and dlom August 8, 2023 06:43

bentito changed the title ~~CCO-372: Azure Workload Identity info in CredsRequests creates a Secret~~ CCO-372: WIP: Azure Workload Identity info in CredsRequests creates a Secret Aug 8, 2023

bentito force-pushed the CCO-372 branch from d736779 to 973abf9 Compare August 9, 2023 05:51

bentito changed the title ~~CCO-372: WIP: Azure Workload Identity info in CredsRequests creates a Secret~~ CCO-372: Azure Workload Identity info in CredsRequests creates a Secret Aug 9, 2023

bentito mentioned this pull request Aug 10, 2023

Add cmd make test-e2e-azident to e2e-azure-manual-oidc openshift/release#42126

Closed

gallettilance force-pushed the CCO-372 branch 2 times, most recently from 833a769 to b5372ca Compare November 28, 2023 16:21

gallettilance force-pushed the CCO-372 branch 3 times, most recently from c39d244 to 1465460 Compare November 28, 2023 21:47

gallettilance force-pushed the CCO-372 branch from 69854e3 to 32cc88b Compare November 29, 2023 16:16

gallettilance reviewed Nov 29, 2023

View reviewed changes

jstuever requested changes Nov 29, 2023

View reviewed changes

Azure workload identity for layered OLM operators

7c970c7

gallettilance force-pushed the CCO-372 branch from 32cc88b to 7c970c7 Compare November 29, 2023 18:11

jstuever approved these changes Nov 29, 2023

View reviewed changes

openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Nov 29, 2023

openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Nov 29, 2023

openshift-merge-bot bot merged commit e295bb2 into openshift:master Nov 29, 2023
12 checks passed

periklis mentioned this pull request Dec 14, 2023

cluster-logging: Add LokiStack tokenized auth proposal openshift/enhancements#1503

Merged

jstuever mentioned this pull request Dec 14, 2023

[release-4.14] Ocpbugs 25275: Azure Workload Identity for layered OLM Operators #641

Closed

openshift-cherrypick-robot mentioned this pull request Dec 18, 2023

[release-4.14] OCPBUGS-25275: Azure Workload Identity info in CredsRequests creates a Secret #643

Merged

CCO-372: Azure Workload Identity info in CredsRequests creates a Secret #587

CCO-372: Azure Workload Identity info in CredsRequests creates a Secret #587

Conversation

bentito commented Aug 2, 2023 • edited

openshift-ci-robot commented Aug 2, 2023 • edited by openshift-ci bot

openshift-ci bot commented Aug 2, 2023

openshift-ci-robot commented Aug 8, 2023 • edited by openshift-ci bot

openshift-ci-robot commented Aug 8, 2023 • edited by openshift-ci bot

openshift-ci-robot commented Aug 8, 2023 • edited by openshift-ci bot

codecov bot commented Aug 8, 2023 • edited

Codecov Report

openshift-ci-robot commented Aug 9, 2023 • edited by openshift-ci bot

openshift-ci-robot commented Aug 9, 2023 • edited by openshift-ci bot

bentito commented Aug 9, 2023

openshift-ci bot commented Aug 9, 2023

bentito commented Aug 9, 2023

openshift-ci-robot commented Aug 9, 2023 • edited by openshift-ci bot

openshift-ci-robot commented Aug 9, 2023 • edited by openshift-ci bot

openshift-ci-robot commented Aug 9, 2023 • edited by openshift-ci bot

bentito commented Aug 9, 2023

bentito commented Aug 9, 2023

bentito commented Aug 9, 2023

openshift-ci-robot commented Aug 9, 2023 • edited by openshift-ci bot

bentito commented Aug 10, 2023

bentito commented Aug 10, 2023 • edited

gallettilance commented Nov 28, 2023

gallettilance commented Nov 28, 2023

gallettilance commented Nov 28, 2023

jstuever commented Nov 28, 2023

gallettilance commented Nov 28, 2023

gallettilance left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jstuever commented Nov 29, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jstuever commented Nov 29, 2023

jstuever commented Nov 29, 2023

jstuever left a comment

Choose a reason for hiding this comment

openshift-ci bot commented Nov 29, 2023

openshift-ci bot commented Nov 29, 2023

openshift-bot commented Nov 30, 2023

gallettilance commented Dec 18, 2023

openshift-cherrypick-robot commented Dec 18, 2023

bentito commented Aug 2, 2023 •

edited

openshift-ci-robot commented Aug 2, 2023 •

edited by openshift-ci bot

openshift-ci-robot commented Aug 8, 2023 •

edited by openshift-ci bot

openshift-ci-robot commented Aug 8, 2023 •

edited by openshift-ci bot

openshift-ci-robot commented Aug 8, 2023 •

edited by openshift-ci bot

codecov bot commented Aug 8, 2023 •

edited

openshift-ci-robot commented Aug 9, 2023 •

edited by openshift-ci bot

openshift-ci-robot commented Aug 9, 2023 •

edited by openshift-ci bot

openshift-ci-robot commented Aug 9, 2023 •

edited by openshift-ci bot

openshift-ci-robot commented Aug 9, 2023 •

edited by openshift-ci bot

openshift-ci-robot commented Aug 9, 2023 •

edited by openshift-ci bot

openshift-ci-robot commented Aug 9, 2023 •

edited by openshift-ci bot

bentito commented Aug 10, 2023 •

edited