Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
x/build: report failures fast but still make all test results available #36181
(This feature request/enhancement is related to issue #34119, the goal is to improve developer experience at the expense of using more computational resources.)
In #14305 (comment), there is discussion of whether to use -k on which branches, and on post-submit only vs trybots too. There is a trade-off in that the flag improves information that a failed build reports, but makes the test failure get reported more slowly.
A more ideal (for user developer experience) solution is to report a failure fast, as soon as the first test fails, but keep going and make all test results available for later viewing by interested parties.
See the fast-finish builds feature of Travis CI for some related precedent.
Right now, on release branches, testing is stopped very soon after encountering the first package with a failing test. For example: [...] FAIL cmd/go/internal/modfetch 17.181s ok cmd/go/internal/modfetch/codehost 5.474s FAIL 2019/11/07 02:35:34 Failed: exit status 1 Ideally we should not have failing tests on release branches for long, but that is not the current state. Until we reach that state, make coordinator run 'go tool dist test' with -k flag so that it keeps testing all remaining packages even if one fails, and report complete test results in the log. That way, a package that fails early doesn't make it completely impossible to see if other package tests pass or fail. Updates golang/go#14305 Updates golang/go#36181 Change-Id: Ia674005ae45c6b9dbffa6f5b56d60b7ed38f6b34 Reviewed-on: https://go-review.googlesource.com/c/build/+/211678 Run-TryBot: Dmitri Shuralyov <firstname.lastname@example.org> TryBot-Result: Gobot Gobot <email@example.com> Reviewed-by: Alexander Rakoczy <firstname.lastname@example.org>