Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/build: ensure that TryBot builders are using a released toolchain for GOROOT_BOOTSTRAP #30633

Open
bcmills opened this issue Mar 6, 2019 · 10 comments

Comments

Projects
None yet
4 participants
@bcmills
Copy link
Member

commented Mar 6, 2019

The OpenBSD builder is using an unreleased development build of the go toolchain:

buildlet$ ./go1.4/bin/go version
go version devel +f14067f3c1 Tue Oct 30 17:45:19 2018 +0000 openbsd/amd64

That builder also runs as a TryBot. That means:

  1. Failures of the builder stand in the way of active Go development (such as #30228).
  2. Such failures are difficult to reproduce, and may be due to bugs fixed after Oct. 30.

Now that Go 1.12 is released, we should make sure that all of the TryBot builders are using a released go toolchain for their GOROOT_BOOTSTRAP.

CC @dmitshur @bradfitz

@gopherbot gopherbot added this to the Unreleased milestone Mar 6, 2019

@gopherbot gopherbot added the Builders label Mar 6, 2019

@bcmills bcmills added OS-OpenBSD and removed Builders labels Mar 6, 2019

@gopherbot gopherbot added the Builders label Mar 6, 2019

@bradfitz

This comment has been minimized.

Copy link
Member

commented Mar 6, 2019

We no longer do releases of Go 1.4. Often times when Go 1.4 no longer bootstraps on a modern version of an OS, we need to use a tip or release-branch.go1.14 commit to get it bootstrapped.

Now that Go 1.12 is released,

What does Go 1.12 have to do with it? The bootstrap builder only needs to be Go 1.4-ish.

@bcmills

This comment has been minimized.

Copy link
Member Author

commented Mar 6, 2019

It doesn't really matter what version the builder uses, as long as it is some version for which we can easily reproduce failures. A released Go 1.11 (if it works) or Go 1.12 should be fine.

The reason 1.12 is relevant is that it presumably supports OpenBSD — meaning that there is at least one supported release that could be used instead of the current mid-cycle development build.

@bradfitz

This comment has been minimized.

Copy link
Member

commented Mar 6, 2019

Such failures are difficult to reproduce,

I've seen your OpenBSD failures keep occuring. It seems that reproducing it isn't the problem here. Presumably gomote will also fail in the same way. I assume you mean locally on Linux? Have you tried using that git rev as your local GOROOT_BOOTSTRAP version on Linux?

But yeah, I agree that the bootstrap tarball shouldn't be from some random revision. Something on release-branch.go1.4 seems preferable (to test our minimum supported version and keep us honest), but Go 1.4 didn't have the cross-compiling bootstrap.bash support for x/build/cmd/genbootstrap. So I guess we can just rebuild from release-branch.go1.12.

I have other things on my plate at the moment so feel free to work on this or we can just keep it open.

@bcmills

This comment has been minimized.

Copy link
Member Author

commented Mar 6, 2019

Have you tried using that git rev as your local GOROOT_BOOTSTRAP version on Linux?

Yeah, and that failure mode is reproducible on Linux at that rev. (Which makes it even more frustrating, since any workaround would be specific to that unreleased version of the toolchain.)

@bradfitz

This comment has been minimized.

Copy link
Member

commented Mar 6, 2019

Great. Sounds like an easy fix then, just re-running genbootstrap & upload a new tarball (with unique filename) & updating the URL in dashboard/builders.go. (Note that the OpenBSD ABI changed some time ago so there was a period where some binaries from certain Go versions wouldn't run on some OS versions, so it's safer to only make new tarballs with unique URLs so we can roll back.)

@gopherbot

This comment has been minimized.

Copy link

commented Mar 6, 2019

Change https://golang.org/cl/165747 mentions this issue: dashboard: set openbsd builders to use a Go 1.12 bootstrap toolchain

gopherbot pushed a commit to golang/build that referenced this issue Mar 6, 2019

dashboard: set openbsd builders to use a Go 1.12 bootstrap toolchain
Updates golang/go#30633

Change-Id: I593a66e18f7546fdeaf7513e2a16bdf82a5fa5a8
Reviewed-on: https://go-review.googlesource.com/c/build/+/165747
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
@4a6f656c

This comment has been minimized.

Copy link
Contributor

commented Mar 27, 2019

The other option here would be to install the Go package that is available for the given OpenBSD release. That way you would be using a more recent version and one that is easily reproducible (e.g. install OpenBSD and run 'pkg_add go'). This also avoids any ABI issues with a Go 1.4 tarball.

@bcmills

This comment has been minimized.

Copy link
Member Author

commented Mar 27, 2019

@4a6f656c, I would really rather we not rely on system-packaged versions of the toolchain.

I don't know about OpenBSD in particular, but packagers on some systems have been known to remove files or otherwise alter the package from the standard go distribution. We should test against the software that we provide, not against a version of that software with (potentially) arbitrary alterations and deletions.

@bcmills

This comment has been minimized.

Copy link
Member Author

commented Apr 12, 2019

Remaining work for this issue is to audit the other builders to determine whether they're all using released versions.

@gopherbot

This comment has been minimized.

Copy link

commented Apr 16, 2019

Change https://golang.org/cl/170863 mentions this issue: cmd/gopherbot: CC triaged issues to owners

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.