Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/build: reverse builder config uniformity #21260

Closed
adams-sarah opened this issue Aug 1, 2017 · 16 comments
Closed

x/build: reverse builder config uniformity #21260

adams-sarah opened this issue Aug 1, 2017 · 16 comments
Assignees
Labels
Milestone

Comments

@adams-sarah
Copy link
Contributor

@adams-sarah adams-sarah commented Aug 1, 2017

Onboarding overhead is very high ATM for builder project.

I'd like to discuss a config uniformity overhaul for the reverse buildlets.
We have been migrating various reverse buildlet Dockerfiles / configurations over to various different stages of "new" piecemeal b/c lack of headcount. eg. uses X version of rundockerbuildlet, uses rundockerbuildlet, are builds hermetic?, uses --reverse (legacy) vs. --reverse-type (latest) flag to cmd/buildlet/buildlet.go

I'd like us document the ideal/latest blessed reverse buildlet config, and push to bring all reverse buildlet configurations up to this config scheme.
I'd also like to discuss the potential for a plan to keep reverse buildlet configurations at latest config scheme in the future.

@gopherbot gopherbot added this to the Unreleased milestone Aug 1, 2017
@gopherbot gopherbot added the Builders label Aug 1, 2017
@adams-sarah adams-sarah self-assigned this Aug 1, 2017
@bradfitz

This comment has been minimized.

Copy link
Member

@bradfitz bradfitz commented Aug 2, 2017

I will start referencing bugs from here.

Here is why the Solaris/illumos builders are not ideal: #15581

@gopherbot

This comment has been minimized.

Copy link

@gopherbot gopherbot commented Aug 2, 2017

Change https://golang.org/cl/52633 mentions this issue: cmd/coordinator: clarify in logs which reverse clients are using legacy config

@gopherbot

This comment has been minimized.

Copy link

@gopherbot gopherbot commented Aug 2, 2017

Change https://golang.org/cl/52650 mentions this issue: cmd/buildlet/stage0: stop using deprecated --reverse flag to buildlet

@bradfitz

This comment has been minimized.

Copy link
Member

@bradfitz bradfitz commented Aug 2, 2017

I deployed CL 52633 to see which hosts are using the old --reverse= argument to buildlet instead of --reverse-type= and found:

      1 host type host-darwin-10_8 (mapped from legacy modes ["darwin-amd64-10_8"])
      1 host type host-linux-mipsle (mapped from legacy modes ["linux-mipsle"])
      1 host type host-linux-mips (mapped from legacy modes ["linux-mips"])
      1 host type host-linux-s390x (mapped from legacy modes ["linux-s390x-ibm"])
      2 host type host-darwin-10_12 (mapped from legacy modes ["darwin-amd64-10_12"])
      2 host type host-solaris-amd64 (mapped from legacy modes ["solaris-amd64-smartosbuildlet"])
      3 host type host-darwin-10_10 (mapped from legacy modes ["darwin-amd64-10_10"])
      3 host type host-linux-arm5spacemonkey (mapped from legacy modes ["linux-arm-arm5spacemonkey"])
      5 host type host-linux-ppc64le-osu (mapped from legacy modes ["linux-ppc64le-buildlet"])
      5 host type host-linux-ppc64-osu (mapped from legacy modes ["linux-ppc64-buildlet"])
     14 host type host-darwin-10_11 (mapped from legacy modes ["darwin-amd64-10_11"])
     50 host type host-linux-arm-scaleway (mapped from legacy modes ["host-linux-arm-scaleway"])

Updating the Scaleway images can happen in https://go-review.googlesource.com/c/52293

A bunch of the others are done in https://go-review.googlesource.com/c/52650 (once ppc64 and arm5-spacemonkey are updated)

Updating Mac images is super hard and tedious and SLOW and error prone. I think we'd be better of keeping the mapping in the coordinator and buildlet for a bit. We can hide it from flag listing if helpful to minimize doc clutter.

MIPS seems to be updateable easily enough.

I also want to figure out a plan for #21191 (speaking of cleanup).

gopherbot pushed a commit to golang/build that referenced this issue Aug 2, 2017
…cy config

We want to kill off the --reverse vs --reverse-host code. This will
point out any clients still using the old way. There might not be any.

Updates golang/go#21260

Change-Id: Ic4b283b02891655ee43f2b7db76088df3f12065a
Reviewed-on: https://go-review.googlesource.com/52633
Reviewed-by: Sarah Adams <shadams@google.com>
gopherbot pushed a commit to golang/build that referenced this issue Aug 2, 2017
Updates golang/go#21260

Change-Id: Ic4d96ef3984017944c5a9af087ea7010e5193f32
Reviewed-on: https://go-review.googlesource.com/52650
Reviewed-by: Jessica Frazelle <jessfraz@google.com>
@bradfitz

This comment has been minimized.

Copy link
Member

@bradfitz bradfitz commented Apr 25, 2019

Latest update on which hosts are still using the legacy ways:

  • host type host-darwin-10_10 (mapped from legacy modes ["darwin-amd64-10_10"])
  • host type host-darwin-10_11 (mapped from legacy modes ["darwin-amd64-10_11"])
  • host type host-darwin-10_12 (mapped from legacy modes ["darwin-amd64-10_12"])
  • host type host-linux-arm-scaleway (mapped from legacy modes ["host-linux-arm-scaleway"])
  • host type host-linux-ppc64-osu (mapped from legacy modes ["linux-ppc64-buildlet"])
  • host type host-linux-s390x (mapped from legacy modes ["linux-s390x-ibm"])
  • host type host-solaris-amd64 (mapped from legacy modes ["solaris-amd64-smartosbuildlet"])

Basically no change since Aug 1 2017, except that we no longer OS X 10.8 and the MIPS builders disappeared.

The one fix seems to be that host-linux-ppc64le-osu and host-linux-ppc64-osu are no longer doing the legacy registration.

Also, macOS 10.14 is at least using the new way. And the Mac VMware cluster is easier to work with now since it was upgraded, so we could fix those old versions at least.

@bradfitz

This comment has been minimized.

Copy link
Member

@bradfitz bradfitz commented Oct 10, 2019

Two of those builders are gone now, so remaining is:

  • host type host-darwin-10_11 (mapped from legacy modes ["darwin-amd64-10_11"])
  • host type host-darwin-10_12 (mapped from legacy modes ["darwin-amd64-10_12"])
  • host type host-linux-arm-scaleway (mapped from legacy modes ["host-linux-arm-scaleway"])
  • host type host-linux-ppc64-osu (mapped from legacy modes ["linux-ppc64-buildlet"])
  • host type host-linux-s390x (mapped from legacy modes ["linux-s390x-ibm"])
@gopherbot

This comment has been minimized.

Copy link

@gopherbot gopherbot commented Oct 11, 2019

Change https://golang.org/cl/200757 mentions this issue: env/linux-ppc64le/osuosl: update NOTES

gopherbot pushed a commit to golang/build that referenced this issue Oct 11, 2019
One of the little ppc64le VMs was lost in the upgrade, and we got one
new beefy one.

Updates golang/go#34830
Updates golang/go#21189
Updates golang/go#21260

Change-Id: I4488d106c5c146c6e55d2d3528913f5507b9d57b
Reviewed-on: https://go-review.googlesource.com/c/build/+/200757
Reviewed-by: Bryan C. Mills <bcmills@google.com>
@gopherbot

This comment has been minimized.

Copy link

@gopherbot gopherbot commented Oct 11, 2019

Change https://golang.org/cl/200820 mentions this issue: env/linux-ppc64{,le}/osuosl: update NOTES

gopherbot pushed a commit to golang/build that referenced this issue Oct 15, 2019
Add details of 3 new big VMs: POWER8 (le), POWER9 (le), and POWER8 (be).

Still not configured, but we have them provisioned.

Updates golang/go#34830
Updates golang/go#21189
Updates golang/go#21260

Change-Id: If648e9966485062cda8131caf2ac5f87ab839fc5
Reviewed-on: https://go-review.googlesource.com/c/build/+/200820
Reviewed-by: Bryan C. Mills <bcmills@google.com>
@gopherbot

This comment has been minimized.

Copy link

@gopherbot gopherbot commented Oct 17, 2019

Change https://golang.org/cl/201637 mentions this issue: all: update linux-ppc64le builders

gopherbot pushed a commit to golang/build that referenced this issue Oct 17, 2019
Fixes golang/go#21189
Updates golang/go#21260
Updates golang/go#34830 (ppc64 big endian remains)

Change-Id: I7ff38edb519bf4f34cd05bb795b6a41f72b10640
Reviewed-on: https://go-review.googlesource.com/c/build/+/201637
Reviewed-by: Carlos Eduardo Seo <cseo@linux.vnet.ibm.com>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
@gopherbot

This comment has been minimized.

Copy link

@gopherbot gopherbot commented Nov 4, 2019

Change https://golang.org/cl/203886 mentions this issue: env/linux-ppc64/osuosl: upgrade linux/ppc64 hardware & OS, use Docker

gopherbot pushed a commit to golang/build that referenced this issue Nov 4, 2019
Collaboration with @tiborvass at Docker who got Docker running on
big-endian PPC64. Go for ppc64 doesn't support cgo or external
linking, so runc doesn't work, but a new OCI-compliant runc
implementation written in C (https://github.com/containers/crun) means
we can run Docker after all. See NOTES & build-*.sh

Then add a Dockerfile & associated cleanup in buildlet & stage0 to use
rundockerbuildlet.

Once done, might help with golang/go#35188, golang/go#32613, etc.

Fixes golang/go#34830
Updates golang/go#21260

Change-Id: I43d7afa1d58bbdfa16e3c57670bc41f1d1932d80
Reviewed-on: https://go-review.googlesource.com/c/build/+/203886
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
@bradfitz

This comment has been minimized.

Copy link
Member

@bradfitz bradfitz commented Nov 4, 2019

Down to:

  • host type host-darwin-10_11 (mapped from legacy modes ["darwin-amd64-10_11"])
  • host type host-darwin-10_12 (mapped from legacy modes ["darwin-amd64-10_12"])
  • host type host-linux-arm-scaleway (mapped from legacy modes ["host-linux-arm-scaleway"])
  • host type host-linux-s390x (mapped from legacy modes ["linux-s390x-ibm"])
@bradfitz

This comment has been minimized.

Copy link
Member

@bradfitz bradfitz commented Nov 5, 2019

Just double checked, and yes:

$ kubectl logs $(k get -o name po -l app=coordinator) | grep "mapped from legacy" | perl -ne 'm/host type (\S+)/ and print "$1\n"' | sort | uniq -c
     29 host-darwin-10_11
    215 host-darwin-10_12
    327 host-linux-arm-scaleway 
    214 host-linux-s390x
@bradfitz

This comment has been minimized.

Copy link
Member

@bradfitz bradfitz commented Nov 5, 2019

And I just fixed the darwin ones.

@gopherbot

This comment has been minimized.

Copy link

@gopherbot gopherbot commented Nov 5, 2019

Change https://golang.org/cl/205357 mentions this issue: cmd/buildlet, cmd/makemac: stop using legacy buildlet --reverse flag for Macs

gopherbot pushed a commit to golang/build that referenced this issue Nov 5, 2019
…for Macs

Updates golang/go#21260

Change-Id: I4e2c61117cfebffd20ab4acd3ee736172c95d153
Reviewed-on: https://go-review.googlesource.com/c/build/+/205357
Reviewed-by: Bryan C. Mills <bcmills@google.com>
@gopherbot

This comment has been minimized.

Copy link

@gopherbot gopherbot commented Nov 6, 2019

Change https://golang.org/cl/205603 mentions this issue: all: upgrade scaleway linux-arm builders, stop using deprecated --reverse flag

@bradfitz bradfitz assigned bradfitz and unassigned adams-sarah Nov 6, 2019
@gopherbot

This comment has been minimized.

Copy link

@gopherbot gopherbot commented Nov 6, 2019

Change https://golang.org/cl/205608 mentions this issue: cmd/buildlet, cmd/coordinator: delete old --reverse mode

gopherbot pushed a commit to golang/build that referenced this issue Nov 6, 2019
…erse flag

Updates golang/go#21260 (no more buildlets using the --reverse flag)
Updates golang/go#33574 (linux-arm kernel+userspace updated)

Change-Id: I7455f6fa3e851f1f9f81d6f1eb487ef7e4bea55b
Reviewed-on: https://go-review.googlesource.com/c/build/+/205603
Reviewed-by: Bryan C. Mills <bcmills@google.com>
codebien added a commit to codebien/build that referenced this issue Nov 13, 2019
One of the little ppc64le VMs was lost in the upgrade, and we got one
new beefy one.

Updates golang/go#34830
Updates golang/go#21189
Updates golang/go#21260

Change-Id: I4488d106c5c146c6e55d2d3528913f5507b9d57b
Reviewed-on: https://go-review.googlesource.com/c/build/+/200757
Reviewed-by: Bryan C. Mills <bcmills@google.com>
codebien added a commit to codebien/build that referenced this issue Nov 13, 2019
Add details of 3 new big VMs: POWER8 (le), POWER9 (le), and POWER8 (be).

Still not configured, but we have them provisioned.

Updates golang/go#34830
Updates golang/go#21189
Updates golang/go#21260

Change-Id: If648e9966485062cda8131caf2ac5f87ab839fc5
Reviewed-on: https://go-review.googlesource.com/c/build/+/200820
Reviewed-by: Bryan C. Mills <bcmills@google.com>
codebien added a commit to codebien/build that referenced this issue Nov 13, 2019
Fixes golang/go#21189
Updates golang/go#21260
Updates golang/go#34830 (ppc64 big endian remains)

Change-Id: I7ff38edb519bf4f34cd05bb795b6a41f72b10640
Reviewed-on: https://go-review.googlesource.com/c/build/+/201637
Reviewed-by: Carlos Eduardo Seo <cseo@linux.vnet.ibm.com>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
codebien added a commit to codebien/build that referenced this issue Nov 13, 2019
Collaboration with @tiborvass at Docker who got Docker running on
big-endian PPC64. Go for ppc64 doesn't support cgo or external
linking, so runc doesn't work, but a new OCI-compliant runc
implementation written in C (https://github.com/containers/crun) means
we can run Docker after all. See NOTES & build-*.sh

Then add a Dockerfile & associated cleanup in buildlet & stage0 to use
rundockerbuildlet.

Once done, might help with golang/go#35188, golang/go#32613, etc.

Fixes golang/go#34830
Updates golang/go#21260

Change-Id: I43d7afa1d58bbdfa16e3c57670bc41f1d1932d80
Reviewed-on: https://go-review.googlesource.com/c/build/+/203886
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
codebien added a commit to codebien/build that referenced this issue Nov 13, 2019
…for Macs

Updates golang/go#21260

Change-Id: I4e2c61117cfebffd20ab4acd3ee736172c95d153
Reviewed-on: https://go-review.googlesource.com/c/build/+/205357
Reviewed-by: Bryan C. Mills <bcmills@google.com>
codebien added a commit to codebien/build that referenced this issue Nov 13, 2019
…erse flag

Updates golang/go#21260 (no more buildlets using the --reverse flag)
Updates golang/go#33574 (linux-arm kernel+userspace updated)

Change-Id: I7455f6fa3e851f1f9f81d6f1eb487ef7e4bea55b
Reviewed-on: https://go-review.googlesource.com/c/build/+/205603
Reviewed-by: Bryan C. Mills <bcmills@google.com>
codebien added a commit to codebien/build that referenced this issue Nov 13, 2019
All the buildlets have been updated to use --reverse-type with a host
type instead of a builder type.

Fixes golang/go#21260

Change-Id: I1264261f099c3686fe01455949486f523b94c6de
Reviewed-on: https://go-review.googlesource.com/c/build/+/205608
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.