Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/build: add OpenBSD 6.4 builder #26833

Closed
bradfitz opened this issue Aug 7, 2018 · 25 comments

Comments

Projects
None yet
6 participants
@bradfitz
Copy link
Member

commented Aug 7, 2018

Update OpenBSD to 6.4.

Would've helped with testing e.g. https://go-review.googlesource.com/c/sys/+/127762.

@gopherbot

This comment has been minimized.

Copy link

commented Sep 4, 2018

Change https://golang.org/cl/133115 mentions this issue: env/openbsd-amd64: use version 6.3, make disklabel readable

gopherbot pushed a commit to golang/build that referenced this issue Sep 4, 2018

env/openbsd-amd64: use version 6.3, make disklabel readable
Updates golang/go#26833

Restrictive root umask prevented ftp running as unprivileged user from reading disklabel file.

Change-Id: I40b2d5b29881ea3733d245f7705cf33c73d4f7dd
GitHub-Last-Rev: 99fd276
GitHub-Pull-Request: #12
Reviewed-on: https://go-review.googlesource.com/133115
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
@4a6f656c

This comment has been minimized.

Copy link
Contributor

commented Sep 5, 2018

Historically we have only updated the OpenBSD builders every two releases, so that we reduce the effort to once every 12 months while keeping on a supported release. As such, I would have expected that we would go from 6.2 to 6.4, when it is released in November.

Is the plan to upgrade to 6.3 now, then 6.4 in November and then follow every release (once every six months) after that?

@bradfitz

This comment has been minimized.

Copy link
Member Author

commented Sep 5, 2018

If we did that historically, I'm not sure it's ever been because of some plan. I think we've just fallen behind and that's what's ended up happening by accident. Ideally we'd have ever version as soon as possible. (This is independent of Go's general OpenBSD support policy; but knowing about breakages sooner will help us prepare.)

@4a6f656c

This comment has been minimized.

Copy link
Contributor

commented Sep 5, 2018

Fair enough - it has been more planned than accidental, as I would normally raise an issue and handle the upgrade work when the builder was about to drop into an unsupported release. Ideally we'd run both currently supported versions so that we detect breakage across either of those...

@bradfitz

This comment has been minimized.

Copy link
Member Author

commented Sep 5, 2018

Sorry, I didn't mean to imply you didn't have a plan. I'm just saying I certainly didn't, or I wasn't aware of one.

@blackgnezdo

This comment has been minimized.

Copy link

commented Sep 19, 2018

I'd start sooner with a 6.4-beta in snapshots if possible. I'm seeing strange behavior on GCE, admittedly with Go 1.10.1 (patched for MAP_STACK).

@dmitshur

This comment has been minimized.

Copy link
Member

commented Sep 21, 2018

I've done some preparatory work, and I'm fairly close to being able to do this. However, it seems that the 6.4 release is due quite soon based on what I'm hearing, and the previous releases:

 • 2018-04-02: BSD Release: OpenBSD 6.3
 • 2017-10-09: BSD Release: OpenBSD 6.2
 • 2017-04-11: BSD Release: OpenBSD 6.1
 • 2016-09-01: BSD Release: OpenBSD 6.0
 • 2016-03-29: BSD Release: OpenBSD 5.9
 • 2015-10-18: BSD Release: OpenBSD 5.8
 • 2015-04-30: BSD Release: OpenBSD 5.7
 • 2014-11-01: BSD Release: OpenBSD 5.6
 • 2014-05-01: BSD Release: OpenBSD 5.5
 • 2013-11-01: BSD Release: OpenBSD 5.4
 • 2013-05-01: BSD Release: OpenBSD 5.3
 • 2012-11-01: BSD Release: OpenBSD 5.2

We're a week away from October now. At this point, it seems worthwhile to wait a little more for 6.4 to come out and target that instead of 6.3. Let me know if there are strong reasons to either go with 6.3 or a beta snapshot of 6.4 sooner.

Otherwise, I plan to resolve this issue as soon as 6.4 final is released. If there is any additional information about the OpenBSD release schedule and when 6.4 final is expected to be released, please let me know.

@blackgnezdo

This comment has been minimized.

Copy link

commented Sep 21, 2018

I would do some practice runs with 6.4-beta currently in snapshots. I made a few minor tweaks to enable running with snapshots for syzkaller. If things don't seem entirely trivial, I'm happy to back-port the changes to make.bash.

@dmitshur

This comment has been minimized.

Copy link
Member

commented Sep 25, 2018

Okay, I'll start with adding a 6.4 beta snapshot builder, and update it to final 6.4 once that's out. It seems that it'll be helpful at getting us more useful debug information sooner.

If things don't seem entirely trivial, I'm happy to back-port the changes to make.bash.

If it's fast for you @blackgnezdo to send a CL to add snapshot support to make.bash and you'd like to do it, that'd be helpful. Feel free to add me as a reviewer. Otherwise, I'll work on that, and make use of google/syzkaller@b117b6e as a reference if needed. Thanks!

@dmitshur dmitshur changed the title x/build: add OpenBSD 6.3 builder x/build: add OpenBSD 6.4 builder (beta snapshot for now, final release when it's out) Sep 25, 2018

@gopherbot

This comment has been minimized.

Copy link

commented Oct 4, 2018

Change https://golang.org/cl/139857 mentions this issue: dashboard, env/openbsd: add OpenBSD 6.4 snapshot builders

@dmitshur

This comment has been minimized.

Copy link
Member

commented Oct 5, 2018

I've started working on this in CL 139857.

@blackgnezdo Can you please take a look at my question https://go-review.googlesource.com/c/build/+/133115/8/env/openbsd-amd64/make.bash#52, knowing the motivation behind that change will help me make progress on this. Thanks!

gopherbot pushed a commit to golang/build that referenced this issue Oct 12, 2018

dashboard, env/openbsd: add OpenBSD 6.4 snapshot builders
This change adds amd64 and 386 OpenBSD 6.4 snapshot builders.
The snapshots are from October 4, 2018:

	/pub/OpenBSD/snapshots/amd64/BUILDINFO:
	Build date: 1538669162 - Thu Oct  4 16:06:02 UTC 2018

	/pub/OpenBSD/snapshots/i386/BUILDINFO:
	Build date: 1538681766 - Thu Oct  4 19:36:06 UTC 2018

These are meant to be replaced with the final version of OpenBSD 6.4
when it is released.

Modify make.bash to support fetching snapshot versions.

The amd64 architecture image was tested with cmd/debugnewvm
and passed all tests successfully.

Updates golang/go#26833.

Change-Id: Iea456605ace08f25d4eba37f9a962e6b9af95369
Reviewed-on: https://go-review.googlesource.com/c/139857
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
@dmitshur

This comment has been minimized.

Copy link
Member

commented Oct 12, 2018

CL 139857 has been merged and cmd/coordinator redeployed. There are openbsd-amd64-64 and openbsd-386-64 builders running now.

It's not the final OpenBSD 6.4 release, and they haven't been tested yet, so they should be considered experimental for a few days until they prove to be reliable. They're not enabled for trybots yet.

@mikioh

This comment has been minimized.

Copy link
Contributor

commented Oct 12, 2018

Not a big deal but having a consistent naming convention might be better. For example, freebsd-amd64-11_2 and openbsd-amd64-6_4 not freebsd-amd64-11_2 vs. openbsd-amd64-64.

@bradfitz bradfitz changed the title x/build: add OpenBSD 6.4 builder (beta snapshot for now, final release when it's out) x/build: add OpenBSD 6.4 builder Oct 18, 2018

@bradfitz

This comment has been minimized.

Copy link
Member Author

commented Oct 18, 2018

OpenBSD 6.4 final is out today: https://www.openbsd.org/64.html

@dmitshur

This comment has been minimized.

Copy link
Member

commented Oct 18, 2018

I'll work on updating the images to 6.4 final.

The current 6.4 snapshots have been quite stable on https://build.golang.org/:

Once the 6.4 final images prove to be equally reliable, we can then make TryBots use them.

@blackgnezdo

This comment has been minimized.

Copy link

commented Oct 18, 2018

FWIW, oct 11 snapshot should be nearly identical to 6.4 release.

@gopherbot

This comment has been minimized.

Copy link

commented Oct 19, 2018

Change https://golang.org/cl/143458 mentions this issue: dashboard: update OpenBSD 6.4 builders to final release

gopherbot pushed a commit to golang/build that referenced this issue Oct 19, 2018

dashboard: update OpenBSD 6.4 builders to final release
Previously, they were running a 6.4 snapshot release from October 4, 2018,
added in CL 139857.

The final release of OpenBSD 6.4 has been released on October 18, 2018,
and can be used now. See https://www.openbsd.org/64.html.

Fixup unbound variable in env/openbsd-amd64/make.bash:

	$ ./make.bash
	./make.bash: line 57: PKG_ADD_OPTIONS: unbound variable

The amd64 architecture image was tested with cmd/debugnewvm
and passed all tests successfully.

Updates golang/go#26833.

Change-Id: I0e3b1ed5aa9b2e624d9afbe46a8a344f88ddcc56
Reviewed-on: https://go-review.googlesource.com/c/143458
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
@dmitshur

This comment has been minimized.

Copy link
Member

commented Oct 19, 2018

I've updated the images and sent CL 143458.

Will redeploy cmd/coordinator later tonight (when it's quieter).

After a few days of no issues, the next step will be to turn on TryBots, then this issue can be considered resolved.

We can also consider cleaning up/removing older OpenBSD builders.

@dmitshur

This comment has been minimized.

Copy link
Member

commented Oct 20, 2018

Will redeploy cmd/coordinator later tonight (when it's quieter).

Version golang/build@24469dd of cmd/coordinator is deployed now.

@gopherbot

This comment has been minimized.

Copy link

commented Oct 25, 2018

Change https://golang.org/cl/144777 mentions this issue: dashboard: switch OpenBSD trybot from 6.2 to 6.4

@dmitshur

This comment has been minimized.

Copy link
Member

commented Nov 2, 2018

The latest status is that I've redeployed cmd/coordinator last night with the fix for #28403. Will let the v2 image OpenBSD 6.4 builders run for a few days to get an idea of their reliability and speed, then we can proceed with switching OpenBSD trybots from 6.2 to 6.4.

@dmitshur

This comment has been minimized.

Copy link
Member

commented Nov 9, 2018

The v2 image appears to be equally stable:

The performance has improved after resolving #28403. See https://go-review.googlesource.com/c/build/+/144777/4..5//COMMIT_MSG.

@dmitshur

This comment has been minimized.

Copy link
Member

commented Nov 9, 2018

Reopening for deploy (likely later tonight, when it's quieter, to avoid disruption to running builds).

@dmitshur dmitshur reopened this Nov 9, 2018

@dmitshur

This comment has been minimized.

Copy link
Member

commented Nov 10, 2018

Deployed coordinator. Trybots will now use OpenBSD 6.4 rather than 6.2. E.g.:

@dmitshur dmitshur closed this Nov 10, 2018

@gopherbot

This comment has been minimized.

Copy link

commented Nov 12, 2018

Change https://golang.org/cl/148888 mentions this issue: dashboard: add OpenBSD 6.4 to BuildConfig.buildSubrepos

gopherbot pushed a commit to golang/build that referenced this issue Nov 12, 2018

dashboard: add OpenBSD 6.4 to BuildConfig.buildSubrepos
By now, OpenBSD 6.4 has proven reliable, so we want it to be used
for building subrepos as well.

Updates golang/go#26833

Change-Id: I026207865514a026f86936140a16dd707f9594ea
Reviewed-on: https://go-review.googlesource.com/c/148888
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.