Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/build: report failures fast but still make all test results available #36181

Open
dmitshur opened this issue Dec 17, 2019 · 1 comment
Open

x/build: report failures fast but still make all test results available #36181

dmitshur opened this issue Dec 17, 2019 · 1 comment

Comments

@dmitshur
Copy link
Member

@dmitshur dmitshur commented Dec 17, 2019

(This feature request/enhancement is related to issue #34119, the goal is to improve developer experience at the expense of using more computational resources.)

In #14305 (comment), there is discussion of whether to use -k on which branches, and on post-submit only vs trybots too. There is a trade-off in that the flag improves information that a failed build reports, but makes the test failure get reported more slowly.

A more ideal (for user developer experience) solution is to report a failure fast, as soon as the first test fails, but keep going and make all test results available for later viewing by interested parties.

See the fast-finish builds feature of Travis CI for some related precedent.

/cc @bradfitz @bcmills @toothrot @cagedmantis

@gopherbot

This comment has been minimized.

Copy link

@gopherbot gopherbot commented Dec 17, 2019

Change https://golang.org/cl/211678 mentions this issue: cmd/coordinator: use 'go tool dist test -k' on release branches

gopherbot pushed a commit to golang/build that referenced this issue Dec 19, 2019
Right now, on release branches, testing is stopped very soon after
encountering the first package with a failing test. For example:

	[...]
	FAIL	cmd/go/internal/modfetch	17.181s
	ok  	cmd/go/internal/modfetch/codehost	5.474s
	FAIL
	2019/11/07 02:35:34 Failed: exit status 1

Ideally we should not have failing tests on release branches for long,
but that is not the current state. Until we reach that state, make
coordinator run 'go tool dist test' with -k flag so that it keeps
testing all remaining packages even if one fails, and report complete
test results in the log.

That way, a package that fails early doesn't make it completely
impossible to see if other package tests pass or fail.

Updates golang/go#14305
Updates golang/go#36181

Change-Id: Ia674005ae45c6b9dbffa6f5b56d60b7ed38f6b34
Reviewed-on: https://go-review.googlesource.com/c/build/+/211678
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.