Bug 1726745: Improve Samples Operator test reporting

[openshift-cherrypick-robot]

This is an automated cherry-pick of #22902

/assign adambkaplan

[adambkaplan]

/hold

Waiting until 4.1.z opens

[adambkaplan]

/lgtm

[openshift-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: adambkaplan, openshift-cherrypick-robot

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• test/extended/OWNERS [adambkaplan]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[openshift-merge-robot]

/bugzilla refresh

[openshift-ci-robot]

@openshift-merge-robot: No Bugzilla bug is referenced in the title of this pull request.
To reference a bug, add 'Bug XXX:' to the title of this pull request and request another bug refresh with /bugzilla refresh.

Details

In response to this:

/bugzilla refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[adambkaplan]

@pweil- @stevekuznetsov are we ok overriding the valid-bug check in some cases? Here we are cherry-picking a fix to the test code so z-stream tests dump the right info if the image ecosystem tests fail (which has always been a source of flakiness).

[pweil-]

@pweil- @stevekuznetsov are we ok overriding the valid-bug check in some cases? Here we are cherry-picking a fix to the test code so z-stream tests dump the right info if the image ecosystem tests fail (which has always been a source of flakiness).

Well I guess I'm wondering why a test-only update would even be a candidate for a back port if it is a name change and not associated with a bug. Since @bparees made that recommendation I'd prefer to get his input. Generally I'd say no, you need a bug or it isn't a back port candidate.

[bparees]

@pweil- the backport is to ensure that our automated tests dump useful information when they fail/flake. It's critical to backport these sort of changes/improvements so our 4.1 CI continues to be as high quality as possible.

This seems like a gap in our new process (requiring BZs to merge non-shipping code changes into 4.1) however the "workaround" would be to open a BZ against the test suite component and use that. There should be no restriction on how many of those changes/BZs go into z-stream branches as, again, it is not shipping code and it does not impact QE's workload.

[bparees]

(it's not a name change, it's a fix in the resource dumping logic to use the correct name of the resource when it attempts to dump the resource. W/o this change the resource dumping logic fails because it can't find the resource it's supposed to dump)

[pweil-]

thanks for the clarification. I understand.

+1 to the idea of no restrictions on non-shipping items. I've been working on a doc on patch management I will include the notion of an override in addition to cherry-pick approval. Having to use a workaround would just be process for the sake of process.

[stevekuznetsov]

You could ask a team lead to add the correct labels and remove the invalid-bug one, or make a bugzilla against the component. I'm not super thrilled about either approach but there is precedent for admins adding labels to manually override other things in Prow (like approval)

[adambkaplan]

I'm not super thrilled about either approach but there is precedent for admins adding labels to manually override other things in Prow (like approval)

You'll also have an audit trail on the PR if a team lead adds the right labels - see above.

[derekwaynecarr]

there is nothing wrong with a bz that says "Improve automated testing in release 4.1 branch for sample operators". please create a bz, link, and we can merge in 4.1.z once present.

[jwforres]

@adambkaplan aside from derek's request that you open a BZ for this, is there a reason it still has the hold on it?

[adambkaplan]

/hold cancel

Nope - now just need the bz

[jwforres]

also please update the PR title to reflect that this change has to do with test updates so the changelog is obvious

[openshift-ci-robot]

@openshift-cherrypick-robot: No Bugzilla bug is referenced in the title of this pull request.
To reference a bug, add 'Bug XXX:' to the title of this pull request and request another bug refresh with /bugzilla refresh.

Details

In response to this:

Bug 1726745 - Improve Samples Operator test reporting

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[adambkaplan]

@jwforres Created BZ

[openshift-ci-robot]

@openshift-cherrypick-robot: This pull request references a valid Bugzilla bug. The bug has been updated to refer to the pull request using the external bug tracker.

Details

In response to this:

Bug 1726745: Improve Samples Operator test reporting

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[openshift-ci-robot]

@openshift-cherrypick-robot: No Bugzilla bug is referenced in the title of this pull request.
To reference a bug, add 'Bug XXX:' to the title of this pull request and request another bug refresh with /bugzilla refresh.

Details

In response to this:

Bug 1726745 - Improve Samples Operator Test Reporting

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[adambkaplan]

/bugzilla refresh

[openshift-ci-robot]

@adambkaplan: This pull request references a valid Bugzilla bug.

Details

In response to this:

/bugzilla refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[openshift-bot]

/retest

Please review the full test history for this PR and help us cut down flakes.