New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/build: reverse builder config uniformity #21260

Open
adams-sarah opened this Issue Aug 1, 2017 · 4 comments

Comments

Projects
None yet
3 participants
@adams-sarah
Contributor

adams-sarah commented Aug 1, 2017

Onboarding overhead is very high ATM for builder project.

I'd like to discuss a config uniformity overhaul for the reverse buildlets.
We have been migrating various reverse buildlet Dockerfiles / configurations over to various different stages of "new" piecemeal b/c lack of headcount. eg. uses X version of rundockerbuildlet, uses rundockerbuildlet, are builds hermetic?, uses --reverse (legacy) vs. --reverse-type (latest) flag to cmd/buildlet/buildlet.go

I'd like us document the ideal/latest blessed reverse buildlet config, and push to bring all reverse buildlet configurations up to this config scheme.
I'd also like to discuss the potential for a plan to keep reverse buildlet configurations at latest config scheme in the future.

@gopherbot gopherbot added this to the Unreleased milestone Aug 1, 2017

@gopherbot gopherbot added the Builders label Aug 1, 2017

@adams-sarah adams-sarah self-assigned this Aug 1, 2017

@bradfitz

This comment has been minimized.

Member

bradfitz commented Aug 2, 2017

I will start referencing bugs from here.

Here is why the Solaris/illumos builders are not ideal: #15581

@gopherbot

This comment has been minimized.

gopherbot commented Aug 2, 2017

Change https://golang.org/cl/52633 mentions this issue: cmd/coordinator: clarify in logs which reverse clients are using legacy config

@gopherbot

This comment has been minimized.

gopherbot commented Aug 2, 2017

Change https://golang.org/cl/52650 mentions this issue: cmd/buildlet/stage0: stop using deprecated --reverse flag to buildlet

@bradfitz

This comment has been minimized.

Member

bradfitz commented Aug 2, 2017

I deployed CL 52633 to see which hosts are using the old --reverse= argument to buildlet instead of --reverse-type= and found:

      1 host type host-darwin-10_8 (mapped from legacy modes ["darwin-amd64-10_8"])
      1 host type host-linux-mipsle (mapped from legacy modes ["linux-mipsle"])
      1 host type host-linux-mips (mapped from legacy modes ["linux-mips"])
      1 host type host-linux-s390x (mapped from legacy modes ["linux-s390x-ibm"])
      2 host type host-darwin-10_12 (mapped from legacy modes ["darwin-amd64-10_12"])
      2 host type host-solaris-amd64 (mapped from legacy modes ["solaris-amd64-smartosbuildlet"])
      3 host type host-darwin-10_10 (mapped from legacy modes ["darwin-amd64-10_10"])
      3 host type host-linux-arm5spacemonkey (mapped from legacy modes ["linux-arm-arm5spacemonkey"])
      5 host type host-linux-ppc64le-osu (mapped from legacy modes ["linux-ppc64le-buildlet"])
      5 host type host-linux-ppc64-osu (mapped from legacy modes ["linux-ppc64-buildlet"])
     14 host type host-darwin-10_11 (mapped from legacy modes ["darwin-amd64-10_11"])
     50 host type host-linux-arm-scaleway (mapped from legacy modes ["host-linux-arm-scaleway"])

Updating the Scaleway images can happen in https://go-review.googlesource.com/c/52293

A bunch of the others are done in https://go-review.googlesource.com/c/52650 (once ppc64 and arm5-spacemonkey are updated)

Updating Mac images is super hard and tedious and SLOW and error prone. I think we'd be better of keeping the mapping in the coordinator and buildlet for a bit. We can hide it from flag listing if helpful to minimize doc clutter.

MIPS seems to be updateable easily enough.

I also want to figure out a plan for #21191 (speaking of cleanup).

gopherbot pushed a commit to golang/build that referenced this issue Aug 2, 2017

cmd/coordinator: clarify in logs which reverse clients are using lega…
…cy config

We want to kill off the --reverse vs --reverse-host code. This will
point out any clients still using the old way. There might not be any.

Updates golang/go#21260

Change-Id: Ic4b283b02891655ee43f2b7db76088df3f12065a
Reviewed-on: https://go-review.googlesource.com/52633
Reviewed-by: Sarah Adams <shadams@google.com>

gopherbot pushed a commit to golang/build that referenced this issue Aug 2, 2017

cmd/buildlet/stage0: stop using deprecated --reverse flag to buildlet
Updates golang/go#21260

Change-Id: Ic4d96ef3984017944c5a9af087ea7010e5193f32
Reviewed-on: https://go-review.googlesource.com/52650
Reviewed-by: Jessica Frazelle <jessfraz@google.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment