x/build: add Fedora builder

[bradfitz]

We have a Debian-sid builder. Also add Fedora?

Would've helped with https://go-review.googlesource.com/c/go/+/127895 (for #26369)

[jcajka]

For the record it would also helped with catching earlier https://go-review.googlesource.com/c/go/+/86541 for #23373
It will help with GC to test integration with latets binutils, glibc, kernel,... (latest versions generally pop sooner in Fedora than in Debian)
I would be happy (as Fedora's GC maintainer) to help with maintaining it and investigating and fixing the build failures.
Can you advise what can I do to help make it happen?

[bradfitz]

Well, adding a Fedora container is trivial & easy for us. But the downside is we'd be using the Linux kernel of https://cloud.google.com/container-optimized-os/ . That's a pretty recent kernel always, but might be configured differently than Fedora.

We could add a full Fedora VM image (as we do for a dozen other non-Linux operating systems/versions), but it's more work. (We need to prepare a script which when run on Linux produces a bootable GCE VM image. See the Plan 9 and BSDs in https://github.com/golang/build/tree/master/env)

[jcajka]

Sorry for longer reply time.

I think that both could be beneficial. For containers I think that fedora:rawhide and fedora:latest should be used to test both against the unstable rawhide and latest stable release. Rawhide more like non-blocking smoke test(it could be broken/stale from time to time).

After taking quick peak at the GCE and the scripts. It doesn't support the regular AWS or openstack image formats, right?(Fedora Cloud images are produced for those.) Is there some summary of what needs GCE and Go CI to run(it would be much easier than revers engineering the scripts)?

[jcajka]

@bradfitz ping

[bradfitz]

After taking quick peak at the GCE and the scripts. It doesn't support the regular AWS or openstack image formats, right?(Fedora Cloud images are produced for those.) Is there some summary of what needs GCE and Go CI to run(it would be much easier than revers engineering the scripts)?

Those would probably work fine with the appropriate yaml cloud init metadata.

An overview of the build system is at https://github.com/golang/build/#go-build-tools but the gist is that a VM gets created and it needs to come up & then download the cmd/buildlet binary and run it. (The buildlet binary should not be baked into the VM image)

Generally we use either the x/build/cmd/buildlet/stage0 binary or just some shell script to do that downloading of the cmd/buildlet binary at start-up. I think shell would work fine here, which would fit into the cloud-init yaml metadata.

For instance, NetBSD (to pick an image somewhat arbitrarily) does this:

commands = [
    """cat >> /etc/rc.local <<EOF
(
  export PATH=/usr/pkg/bin:/usr/pkg/sbin:${PATH}
  export GOROOT_BOOTSTRAP=/usr/pkg/go14
  set -x
  echo 'starting buildlet script'
  netstat -rn
  cat /etc/resolv.conf
  dig metadata.google.internal
  (
    set -e
    curl -o /buildlet \$(curl -H 'Metadata-Flavor: Google' http://metadata.google.internal/computeMetadata/v1/instance/attributes/buildlet-binary-url)
    chmod +x /buildlet
    exec /buildlet
  )
  echo 'giving up'
  sleep 10
  halt -p
)

If you can point us at Fedora cloud images that do the cloud-init stuff & the cloud-init yaml to do the above, that's probably all we need.

[jcajka]

@bradfitz thanks. Fedora cloud images can be got from https://download.fedoraproject.org/pub/fedora/linux/releases/29/Cloud/x86_64/images/ there are Fedora-Cloud-Base-29-1.2.x86_64.qcow2 and Fedora-Cloud-Base-29-1.2.x86_64.raw.xz images along with two vagrant boxes. I will look in to hacking up some cloud init files based on the up mentioned ones. And will try to findout if we have some generic way to get the latest image.

[gopherbot]

Change https://golang.org/cl/175999 mentions this issue: env/linux-x86-fedora: add Fedora builder