Finish saving test results on failure

[johnSchnake]

The conformance image should be saving its results
regardless of the results of the tests. However,
with errexit set, when ginkgo gets test failures
it exits 1 which prevents saving the results
for Sonobuoy to pick up.

Fixes: #76036

/kind bug

Special notes for your reviewer:
Currently working on building this image and testing it works as expected.

Does this PR introduce a user-facing change?:

Ensures the conformance test image saves results before exiting when ginkgo returns non-zero value.

[johnSchnake]

/sig testing
/area conformance

[timothysc]

/approve
/lgtm

Please follow up with cherry-pick PR for 1.14 branch

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: johnSchnake, timothysc

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• cluster/images/conformance/OWNERS [timothysc]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[johnSchnake]

/test pull-kubernetes-integration

[johnSchnake]

Manually confirmed that the patch does fix the error case mentioned in the ticket; a failing ginkgo run still ends up saving the results and getting marked as completed via Sonobuoy.

[johnSchnake]

/test pull-kubernetes-e2e-gce

[ixdy]

/hold

[ixdy]

it'd be better to change the wait line to be something like

wait "$(pgrep ginkgo)" && ret=0 || ret=$?

and then the last line of this script would be

exit ${ret}

[johnSchnake]

To clarify to ensure I understand correctly,

is the reason you want to do that so that it isn't just customized for Sonobuoy (waiting for the done file) but also a consumer could wait for the non-zero exit code for indication of pass/fail?

I guess those two things are at odds:

• if we set errexit and the script fails anywhere results wont be "published" and Sonobuoy would hang
• if we dont set errexit and the script fails anywhere the exit code wont be properly reported and it might report success

If my understanding above is accurate, I agree that the lesser evil is to do as you suggest. A hang would at least be indicative of a problem whereas a false-positive is a larger issue that could go undiscovered.

Also +1/TIL to that particular bash logic; I dont think I'd seen that exact trick used before but it seems pretty nifty.

[ixdy]

we generally like keeping errexit on scripts so that we fail fast on syntax errors or other issues - you're sorta required to handle all error cases, or everything will just fail.

another alternative if we want to make sure logs are always uploaded is to add something like

trap saveResults EXIT

and then you wouldn't even need to explicitly call saveResults - it'd always run on script exit.

[dims]

/lgtm
/hold cancel

[BenTheElder]

/priority important-soon