New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/build: use `go tool dist test -k` on fast builders? #14305

Open
minux opened this Issue Feb 12, 2016 · 2 comments

Comments

Projects
None yet
4 participants
@minux
Member

minux commented Feb 12, 2016

Especially for fast builders, I suggest that we make then use the keepgoing
mode of go tool dist test so that even if a test fails, all the remaining tests
are still tested.

This will be especially useful if a flaky tests fails and then masks the real
problem of a given a CL because the affected tests are not run.

As an example, https://golang.org/cl/19455 changes the runtime, but
the openbsd builder fails a unrelated net tests, and the runtime tests are
not run so we don't actually know if the CL catches any problems on
OpenBSD/386.
http://build.golang.org/log/38e92c1c987f7a7eeb4b6bb418c0d95ed706ba80

I don't know if it's worthwhile to also use the -k for the trybots. I can see
arguments from both sides.

/cc @bradfitz @adg

@minux minux changed the title from x/build: use `go tool dist test -k` on fast builders? to Proposal: x/build: use `go tool dist test -k` on fast builders? Feb 12, 2016

@minux minux added the Proposal label Feb 12, 2016

@minux minux added this to the Proposal milestone Feb 13, 2016

@adg

This comment has been minimized.

Contributor

adg commented Feb 15, 2016

I'm in favor of -k for fast builders. Seems like a good way to get a sense of overall build health.

I don't think trybots should use -k. There should be some deterrent to having flaky builds, and having the trybots fail seems like a good impetus for fixing those flakes.

@bradfitz

This comment has been minimized.

Member

bradfitz commented Aug 15, 2016

The builders also don't use "go tool dist test" by itself anyway. The builder coordinator has its own logic for test sharding and when to stop.

I'm not sure how useful this would be in practice. Usually the first failure is enough and the rest would be noise.

I also want to read errors quickly from the builders and not waste more time finding more errors (especially if they'll be useless).

But as long as it can be done without slowing things done, I'm not opposed to this.

I don't have the time to work on this at the moment, though.

@bradfitz bradfitz changed the title from Proposal: x/build: use `go tool dist test -k` on fast builders? to x/build: use `go tool dist test -k` on fast builders? Aug 15, 2016

@bradfitz bradfitz modified the milestones: Unreleased, Proposal Aug 15, 2016

@gopherbot gopherbot added the Builders label Mar 21, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment