Make logs collection more resilient to failures #11996
Conversation
We already use `script_run` in most of the places instead of `assert_script_run` in case of logs collection to attempt gathering more information even if parts of the system might be broken. However, we use `upload_logs` and `script_output` calls without flags to proceed on failure, which prevent further post_fail_hook steps from the execution. See [poo#77161](https://progress.opensuse.org/issues/77161).
For the generic case, we do not want to stop post_fail_hook execution if coredumps cannot be collected. By introducing additional parameter, we can select the behavior, as subroutine is also used in the testing module, where we expect it to fail in case of issues.
|
I wonder if we should make every call within the post_fail_hook non-fatal to try to continue but then again a lot more failed calls can be confusing to test reviewers as well, hm … |
I also had same idea, but I believe that there are cases where we want to stop, if something is terribly wrong and we know that it's waste of time. E.g. in case of explicit |
We already use
script_runin most of the places instead ofassert_script_runin case of logs collection to attempt gathering moreinformation even if parts of the system might be broken.
However, we use
upload_logsandscript_outputcalls without flags toproceed on failure, which prevent further post_fail_hook steps from the
execution.
See poo#77161.
Verification runs
Here we can see that we even have a mechanism to recover network in trivial cases (I've added command to turn off the interface), but previously we would not even attempt it, as one of the upload commands will fail.