x/build: report failures fast but still make all test results available #36181
Labels
Builders
x/build issues (builders, bots, dashboards)
FeatureRequest
NeedsInvestigation
Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone
Comments
dmitshur
added
Builders
|
Change https://golang.org/cl/211678 mentions this issue: |
gopherbot
pushed a commit
to golang/build
that referenced
this issue
Dec 19, 2019
Right now, on release branches, testing is stopped very soon after encountering the first package with a failing test. For example: [...] FAIL cmd/go/internal/modfetch 17.181s ok cmd/go/internal/modfetch/codehost 5.474s FAIL 2019/11/07 02:35:34 Failed: exit status 1 Ideally we should not have failing tests on release branches for long, but that is not the current state. Until we reach that state, make coordinator run 'go tool dist test' with -k flag so that it keeps testing all remaining packages even if one fails, and report complete test results in the log. That way, a package that fails early doesn't make it completely impossible to see if other package tests pass or fail. Updates golang/go#14305 Updates golang/go#36181 Change-Id: Ia674005ae45c6b9dbffa6f5b56d60b7ed38f6b34 Reviewed-on: https://go-review.googlesource.com/c/build/+/211678 Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
|
/cc @findleyr |
|
Change https://golang.org/cl/223780 mentions this issue: |
|
Change https://golang.org/cl/223782 mentions this issue: |
gopherbot
pushed a commit
that referenced
this issue
Mar 31, 2020
…indows This test was fixed by changing cmd/doc behavior in CL 204442. Backporting that non-test code change is unlikely to be appropriate this late in Go 1.13 release cycle. A failing test can cover up other regressions, so skip this known failing test to fix the builder. For #35236. For #36181. Change-Id: I07e795e75d7e37bc96ab68607d5d5cc9254342f8 Reviewed-on: https://go-review.googlesource.com/c/go/+/223780 Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
gopherbot
pushed a commit
that referenced
this issue
Mar 31, 2020
These non-short Windows test failures were resolved fully in CL 206144. Both TestScript/build_trimpath and TestScript/version tests can be fixed by backporting the changes to test scripts only, so that is done here. Fixing TestScript/mod_list_dir requires backporting non-test changes in addition to the test script changes, which is unlikely to be appropriate this late in Go 1.13 release cycle. A failing test can cover up other regressions, so skip this known failing test to fix the builder. For #36181. Change-Id: I4f140bd373554eb4664f04638666dee77986ec3e Reviewed-on: https://go-review.googlesource.com/c/go/+/223782 Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
(This feature request/enhancement is related to issue #34119, the goal is to improve developer experience at the expense of using more computational resources.)
In #14305 (comment), there is discussion of whether to use -k on which branches, and on post-submit only vs trybots too. There is a trade-off in that the flag improves information that a failed build reports, but makes the test failure get reported more slowly.
A more ideal (for user developer experience) solution is to report a failure fast, as soon as the first test fails, but keep going and make all test results available for later viewing by interested parties.
See the fast-finish builds feature of Travis CI for some related precedent.
/cc @bradfitz @bcmills @toothrot @cagedmantis
The text was updated successfully, but these errors were encountered: