feature: `OnError` mode to cancel in-progress TaskRuns

[aThorp96]

Feature request

Add support for an OnError mode which will not only skip future TaskRuns, but will cancel any TaskRuns which are actively running.

Use case

Given a pipeline with two Tasks which might be long-running and start at the same time, e.g. two suites of end to end tests, if one of them fails after 10 minutes and the other may succeed after an hour, any tasks which have not started will be skipped, but I don't want my other end to end test to continue running for an hour since the pipelinerun will ultimately fail.

The onError mode stopAndFail suggests that running-tasks would be stopped/cancelled, but if that is not the correct semantics of stopAndFail then this request is to support a new onError option (not sure what would be a more appropriate name than stopAndFail) which would behave like stopAndFail, but also cancel all running TaskRuns associated with the failed PipelineRun.

[aThorp96]

Discussed in the Working Group call and landed on the following points:

• The current semantics of onError: stopAndFail is correct, and aligns with the PipelineRun field spec.status: StoppedRunFinally as described here.
• There is value in including an OnError value which aligns with the pipelineRun field spec.status: Cancelled and/or spec.status: CancelledRunningFunally as described here.
• This would require a TEP