New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/build: Kubernetes builders on GKE don't support privileged pods #12815

Open
bradfitz opened this Issue Oct 1, 2015 · 4 comments

Comments

Projects
None yet
4 participants
@bradfitz
Member

bradfitz commented Oct 1, 2015

Kubernetes builders on GKE (Google Container Engine) don't support privileged pods.

This means our builds run as user not-really-but-kinda root. They can do lots of things root normally can, but they can't run some of the Linux exec tests.

If we don't run as root, the buildlet has a bunch of misc problems, so it's easier for us to run as root for now and skip some tests.

This bug exists to explain the situation and track CLs which skip tests on Kubernetes (environment: IN_KUBERNETES == "1")

When GKE fixes things so users can run privileged pods we can remove the skips.

We won't be losing any test coverage overall, though, because we'll keep some Linux builders still running as VMs as root. Only the trybots will use Kubernetes with that test skipped for now.

/cc @evandbrown

@bradfitz bradfitz added the Builders label Oct 1, 2015

@bradfitz bradfitz self-assigned this Oct 1, 2015

@gopherbot

This comment has been minimized.

gopherbot commented Oct 1, 2015

CL https://golang.org/cl/15283 mentions this issue.

@minux

This comment has been minimized.

Member

minux commented Oct 2, 2015

@bradfitz

This comment has been minimized.

Member

bradfitz commented Oct 2, 2015

@minux, the buildlet lacks some permissions (starting with unable to listen to port 80) when not run as root. I don't want to debug that at the same time. We'll get a mix of root and not root at some point. I believe a bug is already open for that.

bradfitz added a commit that referenced this issue Oct 2, 2015

syscall: skip a couple tests when running under Kubernetes
Update #12815

Change-Id: I3bf6de74bc8ab07000fe9a4308299839ef20632f
Reviewed-on: https://go-review.googlesource.com/15283
Reviewed-by: Evan Brown <evanbrown@google.com>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>

@rsc rsc added this to the Unreleased milestone Oct 23, 2015

@bradfitz

This comment has been minimized.

Member

bradfitz commented May 10, 2018

This is no longer relevant since we're moving off Kubernetes builders back to VMs (#25108).

But we can keep this bug open track removing the IN_KUBERNETES=1 environment & test skips.

/cc @bcmills @andybons

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment