Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Can't run Bazel as part of a Docker build due to sandboxing issues #418

Closed
ekuefler opened this issue Sep 2, 2015 · 9 comments
Closed

Can't run Bazel as part of a Docker build due to sandboxing issues #418

ekuefler opened this issue Sep 2, 2015 · 9 comments
Assignees
@philwo
Labels
category: sandboxing P4 This is either out of scope or we don't have bandwidth to review a PR. (No assignee) type: feature request

Comments

@ekuefler
Copy link
Contributor

ekuefler commented Sep 2, 2015

I have a Docker image that builds the Bazel completion script. Building this image now fails with the following error:

src/main/tools/namespace-sandbox.c:545: mount("none", "/", NULL, MS_REC | MS_PRIVATE, NULL): Permission denied

An easy workaround is to add --spawn_strategy=standalone --genrule_strategy=standalone to the bazel build command. Not sure if it's intended / feasible for this use case to work with sandboxing.

Full output is as follows:

Step 8 : RUN cd /opt/bazel     && bazel --batch build //scripts:bash_completion     && mv bazel-bin/scripts/bazel-complete.bash /root/.bazel-complete.bash     && rm -rf /root/.cache/
 ---> Running in b9dc77de4eb8
Extracting Bazel installation...
Sep 02, 2015 11:34:26 PM com.google.devtools.build.lib.analysis.BlazeVersionInfo logVersionInfo
INFO: Blaze version info: Build label: head (@8a61174)
Build target: bazel-out/local_linux-fastbuild/bin/src/main/java/bazel-main_deploy.jar
Build time: Wed Sep 2 21:53:53 2015 (1441230833)
Build timestamp: 1441230833
Build timestamp as int: 1441230833
Sep 02, 2015 11:34:26 PM com.google.devtools.build.lib.runtime.BlazeRuntime batchMain
INFO: Running Blaze in batch mode with startup args [--batch, --install_base=/root/.cache/bazel/_bazel_root/install/2250d99e177354339d04026e1bbafe4b, --install_md5=2250d99e177354339d04026e1bbafe4b, --output_base=/root/.cache/bazel/_bazel_root/aef1e07eca3d4d4bac211b6ac3bc513a, --workspace_directory=/opt/bazel, --nofatal_event_bus_exceptions, --option_sources=batch:]
____Loading package: scripts
____Loading package: tools/defaults
____Loading package: src/main/tools
____Loading package: src/main/java
____Loading package: src/main/java/com/google/devtools/build/lib/worker
____Loading package: third_party
____Loading complete.  Analyzing...
____Found 1 target...
____Building...
____[0 / 2] BazelWorkspaceStatusAction stable-status.txt
____[1 / 82] Writing file build-info-redacted.properties
____[6 / 210] Writing file src/main/protobuf/libproto_build.jar-2.params [for host]
____[11 / 210] Writing file src/main/java/libbuildtool-runtime.jar-2.params [for host]
ERROR: Sandboxed execution failed: //src/main/native:copy_link_jni_md_header..
____From Executing genrule //src/main/native:copy_link_jni_md_header [for host]:
src/main/tools/namespace-sandbox.c:545: mount("none", "/", NULL, MS_REC | MS_PRIVATE, NULL): Permission denied
ERROR: /opt/bazel/src/main/native/BUILD:1:1: Executing genrule //src/main/native:copy_link_jni_md_header failed: Error during execution of spawn: Process exited with status 1.
Target //scripts:bash_completion failed to build
Use --verbose_failures to see the command lines of failed build steps.
____Elapsed time: 3.046s, Critical Path: 1.26s
The command '/bin/sh -c cd /opt/bazel     && bazel --batch build //scripts:bash_completion     && mv bazel-bin/scripts/bazel-complete.bash /root/.bazel-complete.bash     && rm -rf /root/.cache/' returned a non-zero code: 1
@ekuefler
Copy link
Contributor Author

ekuefler commented Sep 3, 2015

I think the scope of this is broader than I realized. I'm seeing some of my own builds inside docker containers fail with the same error message, but it's not all of them, and I haven't been able to isolate it yet.

@philwo
Copy link
Member

philwo commented Sep 3, 2015

To understand this correctly - you're trying to run Bazel inside Docker, right?

I think Docker by default blocks certain operations inside containers, which for example means, that you also can't simply run Docker inside Docker. To allow that, it seems like one has to use docker run --privileged ... to give the container the necessary permissions.

Could you see if using the --privileged flag makes Bazel work inside Docker? If yes, we can try to figure out what exact capabilities we need.

FWIW, it seems like this comment describes exactly the situation that we're seeing here: moby/moby#6687 (comment)

@philwo philwo self-assigned this Sep 3, 2015
@ekuefler
Copy link
Contributor Author

ekuefler commented Sep 4, 2015

Okay, dug into this a bit more. There are a few issues layered on top of one another. #134 is closely related.

I defined a Dockerfile in https://gist.github.com/ekuefler/a9a6ffea74966d277503 that does a minimal bazel installation. I can build this Dockerfile on OSX but not on Linux for the reasons described in #134 (endless "Sending SIGTERM to previous Bazel server"). So that's problem number 1.

Having built that image, I can run it on Linux or OSX. I'm testing it by running bazel --batch build //scripts:bash_completion inside the container in various environments:

  • OSX, unprivileged - works
  • Linux, unprivileged - fails with mount("none", "/", NULL, MS_REC | MS_PRIVATE, NULL): Permission denied
  • Linux, privileged - works

So the --privileged flag does fix things, but even ignoring the security concerns it isn't an acceptable workaround. The problem is that docker build always runs in an unprivileged container, so it's not possible to build a docker image on Linux if that image runs bazel as one of its build steps (though it works on OSX).

@damienmg
Copy link
Contributor

damienmg commented Sep 7, 2015

For the mount issue, just deactivate the sandbox strategy is the solution then.

Philip: is it feasible as part of the sandbox detection to test if we have enough privilege?

@dcreager
Copy link

Combining this comment from #134 with @ekuefler's original workaround above, I was able to pass in some extra options via bazelrc, which let me build bazel successfully inside of an unprivileged container on Linux:

[ inside Docker container ]
$ cd /tmp
$ git clone https://github.com/bazelbuild/bazel.git
$ cat >/tmp/bazelrc <<EOF
startup --batch
build --spawn_strategy=standalone --genrule_strategy=standalone
EOF
$ cd /tmp/bazel
$ BAZELRC=/tmp/bazelrc ./compile.sh
[ snip ]
Build successful! Binary is here: /tmp/bazel/output/bazel

@damienmg damienmg added category: sandboxing type: feature request P4 This is either out of scope or we don't have bandwidth to review a PR. (No assignee) platform: other labels Nov 17, 2015
@mnett mnett mentioned this issue Jan 17, 2016
Closed
@philwo
Copy link
Member

philwo commented Oct 13, 2016

The new sandbox implementation has a fallback mechanism that ensures it works (somewhat) on systems where we can't use user / mount namespaces. Workarounds like --spawn_strategy=standalone should not be necessary anymore - if you find it still doesn't work, please report back. In that case we might have to tweak the detection mechanism for the fallback mode a bit :)

@philwo philwo closed this as completed Oct 13, 2016
@mikedanese
Copy link

@philwo Is there a warning if the sandbox fails and bazel falls back? Also I'm still having trouble with this using bazel from the tip of master:

root@f45b80e82cb5:/src# bazel version
Build label: 0.3.2-2016-10-13 (@2891ec5)
Build target: bazel-out/local-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Fri Oct 14 01:41:08 2016 (1476409268)
Build timestamp: 1476409268
Build timestamp as int: 1476409268
root@f45b80e82cb5:/src# bazel build //build:full --verbose_failures --sandbox_debug
INFO: Found 1 target...
ERROR: /src/vendor/BUILD:6752:1: null failed: linux-sandbox failed: error executing command 
  (cd /root/.cache/bazel/_bazel_root/f8087e59fd95af1ae29e8fcb7ff1a3dc/bazel-sandbox/e06c987a-aa7d-4202-ad89-57c4176059cf-3/execroot/src && \
  exec env - \
    GOARCH=amd64 \
    GOOS=linux \
  /root/.cache/bazel/_bazel_root/f8087e59fd95af1ae29e8fcb7ff1a3dc/execroot/src/_bin/linux-sandbox @/root/.cache/bazel/_bazel_root/f8087e59fd95af1ae29e8fcb7ff1a3dc/bazel-sandbox/e06c987a-aa7d-4202-ad89-57c4176059cf-3/linux-sandbox.params -- bazel-out/local-fastbuild/bin/vendor/bazel-out/local-fastbuild/bin/vendor/golang.org/x/net/context.a.GoCompileFile.params).
open k8s.io/kubernetes/vendor/golang.org/x/net/context/context.go: open k8s.io/kubernetes/vendor/golang.org/x/net/context/context.go: permission denied
src/main/tools/linux-sandbox.cc:183: linux-sandbox-pid1 has PID 11884
src/main/tools/linux-sandbox-pid1.cc:241: tmpfs: /tmp
src/main/tools/linux-sandbox-pid1.cc:251: working dir: /root/.cache/bazel/_bazel_root/f8087e59fd95af1ae29e8fcb7ff1a3dc/bazel-sandbox/e06c987a-aa7d-4202-ad89-57c4176059cf-3/execroot/src
src/main/tools/linux-sandbox-pid1.cc:181: CreateTarget(root/.cache/bazel/_bazel_root/f8087e59fd95af1ae29e8fcb7ff1a3dc/bazel-sandbox/e06c987a-aa7d-4202-ad89-57c4176059cf-3/execroot/src, true)
src/main/tools/linux-sandbox-pid1.cc:369: remount ro: /root/.cache/bazel/_bazel_root/f8087e59fd95af1ae29e8fcb7ff1a3dc/bazel-sandbox/e06c987a-aa7d-4202-ad89-57c4176059cf-3/tmp
src/main/tools/linux-sandbox-pid1.cc:369: remount ro: /root/.cache/bazel/_bazel_root/f8087e59fd95af1ae29e8fcb7ff1a3dc/bazel-sandbox/e06c987a-aa7d-4202-ad89-57c4176059cf-3/tmp/proc
src/main/tools/linux-sandbox-pid1.cc:369: remount ro: /root/.cache/bazel/_bazel_root/f8087e59fd95af1ae29e8fcb7ff1a3dc/bazel-sandbox/e06c987a-aa7d-4202-ad89-57c4176059cf-3/tmp/dev
src/main/tools/linux-sandbox-pid1.cc:369: remount ro: /root/.cache/bazel/_bazel_root/f8087e59fd95af1ae29e8fcb7ff1a3dc/bazel-sandbox/e06c987a-aa7d-4202-ad89-57c4176059cf-3/tmp/dev/pts
src/main/tools/linux-sandbox-pid1.cc:369: remount ro: /root/.cache/bazel/_bazel_root/f8087e59fd95af1ae29e8fcb7ff1a3dc/bazel-sandbox/e06c987a-aa7d-4202-ad89-57c4176059cf-3/tmp/dev/shm
src/main/tools/linux-sandbox-pid1.cc:369: remount ro: /root/.cache/bazel/_bazel_root/f8087e59fd95af1ae29e8fcb7ff1a3dc/bazel-sandbox/e06c987a-aa7d-4202-ad89-57c4176059cf-3/tmp/dev/mqueue
src/main/tools/linux-sandbox-pid1.cc:369: remount ro: /root/.cache/bazel/_bazel_root/f8087e59fd95af1ae29e8fcb7ff1a3dc/bazel-sandbox/e06c987a-aa7d-4202-ad89-57c4176059cf-3/tmp/dev/console
src/main/tools/linux-sandbox-pid1.cc:369: remount ro: /root/.cache/bazel/_bazel_root/f8087e59fd95af1ae29e8fcb7ff1a3dc/bazel-sandbox/e06c987a-aa7d-4202-ad89-57c4176059cf-3/tmp/sys
src/main/tools/linux-sandbox-pid1.cc:369: remount ro: /root/.cache/bazel/_bazel_root/f8087e59fd95af1ae29e8fcb7ff1a3dc/bazel-sandbox/e06c987a-aa7d-4202-ad89-57c4176059cf-3/tmp/sys/fs/cgroup
src/main/tools/linux-sandbox-pid1.cc:369: remount ro: /root/.cache/bazel/_bazel_root/f8087e59fd95af1ae29e8fcb7ff1a3dc/bazel-sandbox/e06c987a-aa7d-4202-ad89-57c4176059cf-3/tmp/sys/fs/cgroup/cpuset
src/main/tools/linux-sandbox-pid1.cc:369: remount ro: /root/.cache/bazel/_bazel_root/f8087e59fd95af1ae29e8fcb7ff1a3dc/bazel-sandbox/e06c987a-aa7d-4202-ad89-57c4176059cf-3/tmp/sys/fs/cgroup/cpu
src/main/tools/linux-sandbox-pid1.cc:369: remount ro: /root/.cache/bazel/_bazel_root/f8087e59fd95af1ae29e8fcb7ff1a3dc/bazel-sandbox/e06c987a-aa7d-4202-ad89-57c4176059cf-3/tmp/sys/fs/cgroup/cpuacct
src/main/tools/linux-sandbox-pid1.cc:369: remount ro: /root/.cache/bazel/_bazel_root/f8087e59fd95af1ae29e8fcb7ff1a3dc/bazel-sandbox/e06c987a-aa7d-4202-ad89-57c4176059cf-3/tmp/sys/fs/cgroup/memory
src/main/tools/linux-sandbox-pid1.cc:369: remount ro: /root/.cache/bazel/_bazel_root/f8087e59fd95af1ae29e8fcb7ff1a3dc/bazel-sandbox/e06c987a-aa7d-4202-ad89-57c4176059cf-3/tmp/sys/fs/cgroup/devices
src/main/tools/linux-sandbox-pid1.cc:369: remount ro: /root/.cache/bazel/_bazel_root/f8087e59fd95af1ae29e8fcb7ff1a3dc/bazel-sandbox/e06c987a-aa7d-4202-ad89-57c4176059cf-3/tmp/sys/fs/cgroup/freezer
src/main/tools/linux-sandbox-pid1.cc:369: remount ro: /root/.cache/bazel/_bazel_root/f8087e59fd95af1ae29e8fcb7ff1a3dc/bazel-sandbox/e06c987a-aa7d-4202-ad89-57c4176059cf-3/tmp/sys/fs/cgroup/blkio
src/main/tools/linux-sandbox-pid1.cc:369: remount ro: /root/.cache/bazel/_bazel_root/f8087e59fd95af1ae29e8fcb7ff1a3dc/bazel-sandbox/e06c987a-aa7d-4202-ad89-57c4176059cf-3/tmp/sys/fs/cgroup/perf_event
src/main/tools/linux-sandbox-pid1.cc:369: remount ro: /root/.cache/bazel/_bazel_root/f8087e59fd95af1ae29e8fcb7ff1a3dc/bazel-sandbox/e06c987a-aa7d-4202-ad89-57c4176059cf-3/tmp/sys/fs/cgroup/hugetlb
src/main/tools/linux-sandbox-pid1.cc:369: remount ro: /root/.cache/bazel/_bazel_root/f8087e59fd95af1ae29e8fcb7ff1a3dc/bazel-sandbox/e06c987a-aa7d-4202-ad89-57c4176059cf-3/tmp/sys/fs/cgroup/systemd
src/main/tools/linux-sandbox-pid1.cc:369: remount ro: /root/.cache/bazel/_bazel_root/f8087e59fd95af1ae29e8fcb7ff1a3dc/bazel-sandbox/e06c987a-aa7d-4202-ad89-57c4176059cf-3/tmp/src
src/main/tools/linux-sandbox-pid1.cc:369: remount ro: /root/.cache/bazel/_bazel_root/f8087e59fd95af1ae29e8fcb7ff1a3dc/bazel-sandbox/e06c987a-aa7d-4202-ad89-57c4176059cf-3/tmp/root/.cache/bazel
src/main/tools/linux-sandbox-pid1.cc:369: remount ro: /root/.cache/bazel/_bazel_root/f8087e59fd95af1ae29e8fcb7ff1a3dc/bazel-sandbox/e06c987a-aa7d-4202-ad89-57c4176059cf-3/tmp/etc/resolv.conf
src/main/tools/linux-sandbox-pid1.cc:369: remount ro: /root/.cache/bazel/_bazel_root/f8087e59fd95af1ae29e8fcb7ff1a3dc/bazel-sandbox/e06c987a-aa7d-4202-ad89-57c4176059cf-3/tmp/etc/hostname
src/main/tools/linux-sandbox-pid1.cc:369: remount ro: /root/.cache/bazel/_bazel_root/f8087e59fd95af1ae29e8fcb7ff1a3dc/bazel-sandbox/e06c987a-aa7d-4202-ad89-57c4176059cf-3/tmp/etc/hosts
src/main/tools/linux-sandbox-pid1.cc:369: remount rw: /root/.cache/bazel/_bazel_root/f8087e59fd95af1ae29e8fcb7ff1a3dc/bazel-sandbox/e06c987a-aa7d-4202-ad89-57c4176059cf-3/tmp/tmp
src/main/tools/linux-sandbox-pid1.cc:369: remount rw: /root/.cache/bazel/_bazel_root/f8087e59fd95af1ae29e8fcb7ff1a3dc/bazel-sandbox/e06c987a-aa7d-4202-ad89-57c4176059cf-3/tmp/root/.cache/bazel/_bazel_root/f8087e59fd95af1ae29e8fcb7ff1a3dc/bazel-sandbox/e06c987a-aa7d-4202-ad89-57c4176059cf-3/execroot/src
src/main/tools/linux-sandbox-pid1.cc:477: sigaction(32, &sa, NULL) failed
src/main/tools/linux-sandbox-pid1.cc:477: sigaction(33, &sa, NULL) failed
src/main/tools/linux-sandbox-pid1.cc:587: waitpid returned 2
src/main/tools/linux-sandbox-pid1.cc:607: child exited with code 1
src/main/tools/linux-sandbox.cc:223: child exited normally with exitcode 1
Target //build:full failed to build

and

root@573517f55369:/src#  /root/.cache/bazel/_bazel_root/f8087e59fd95af1ae29e8fcb7ff1a3dc/execroot/src/_bin/linux-sandbox @/root/.cache/bazel/_bazel_root/f8087e59fd95af1ae29e8fcb7ff1a3dc/bazel-sandbox/143ece43-e9c8-4770-921e-152b0ee6e079-0/linux-sandbox.params --  bash
src/main/tools/linux-sandbox.cc:183: linux-sandbox-pid1 has PID 8352
src/main/tools/linux-sandbox-pid1.cc:241: tmpfs: /tmp
src/main/tools/linux-sandbox-pid1.cc:251: working dir: /root/.cache/bazel/_bazel_root/f8087e59fd95af1ae29e8fcb7ff1a3dc/bazel-sandbox/143ece43-e9c8-4770-921e-152b0ee6e079-0/execroot/src
src/main/tools/linux-sandbox-pid1.cc:181: CreateTarget(root/.cache/bazel/_bazel_root/f8087e59fd95af1ae29e8fcb7ff1a3dc/bazel-sandbox/143ece43-e9c8-4770-921e-152b0ee6e079-0/execroot/src, true)
src/main/tools/linux-sandbox-pid1.cc:369: remount ro: /root/.cache/bazel/_bazel_root/f8087e59fd95af1ae29e8fcb7ff1a3dc/bazel-sandbox/143ece43-e9c8-4770-921e-152b0ee6e079-0/tmp
src/main/tools/linux-sandbox-pid1.cc:369: remount ro: /root/.cache/bazel/_bazel_root/f8087e59fd95af1ae29e8fcb7ff1a3dc/bazel-sandbox/143ece43-e9c8-4770-921e-152b0ee6e079-0/tmp/proc
src/main/tools/linux-sandbox-pid1.cc:369: remount ro: /root/.cache/bazel/_bazel_root/f8087e59fd95af1ae29e8fcb7ff1a3dc/bazel-sandbox/143ece43-e9c8-4770-921e-152b0ee6e079-0/tmp/dev
src/main/tools/linux-sandbox-pid1.cc:369: remount ro: /root/.cache/bazel/_bazel_root/f8087e59fd95af1ae29e8fcb7ff1a3dc/bazel-sandbox/143ece43-e9c8-4770-921e-152b0ee6e079-0/tmp/dev/pts
src/main/tools/linux-sandbox-pid1.cc:369: remount ro: /root/.cache/bazel/_bazel_root/f8087e59fd95af1ae29e8fcb7ff1a3dc/bazel-sandbox/143ece43-e9c8-4770-921e-152b0ee6e079-0/tmp/dev/shm
src/main/tools/linux-sandbox-pid1.cc:369: remount ro: /root/.cache/bazel/_bazel_root/f8087e59fd95af1ae29e8fcb7ff1a3dc/bazel-sandbox/143ece43-e9c8-4770-921e-152b0ee6e079-0/tmp/dev/mqueue
src/main/tools/linux-sandbox-pid1.cc:369: remount ro: /root/.cache/bazel/_bazel_root/f8087e59fd95af1ae29e8fcb7ff1a3dc/bazel-sandbox/143ece43-e9c8-4770-921e-152b0ee6e079-0/tmp/dev/console
src/main/tools/linux-sandbox-pid1.cc:369: remount ro: /root/.cache/bazel/_bazel_root/f8087e59fd95af1ae29e8fcb7ff1a3dc/bazel-sandbox/143ece43-e9c8-4770-921e-152b0ee6e079-0/tmp/sys
src/main/tools/linux-sandbox-pid1.cc:369: remount ro: /root/.cache/bazel/_bazel_root/f8087e59fd95af1ae29e8fcb7ff1a3dc/bazel-sandbox/143ece43-e9c8-4770-921e-152b0ee6e079-0/tmp/sys/fs/cgroup
src/main/tools/linux-sandbox-pid1.cc:369: remount ro: /root/.cache/bazel/_bazel_root/f8087e59fd95af1ae29e8fcb7ff1a3dc/bazel-sandbox/143ece43-e9c8-4770-921e-152b0ee6e079-0/tmp/sys/fs/cgroup/cpuset
src/main/tools/linux-sandbox-pid1.cc:369: remount ro: /root/.cache/bazel/_bazel_root/f8087e59fd95af1ae29e8fcb7ff1a3dc/bazel-sandbox/143ece43-e9c8-4770-921e-152b0ee6e079-0/tmp/sys/fs/cgroup/cpu
src/main/tools/linux-sandbox-pid1.cc:369: remount ro: /root/.cache/bazel/_bazel_root/f8087e59fd95af1ae29e8fcb7ff1a3dc/bazel-sandbox/143ece43-e9c8-4770-921e-152b0ee6e079-0/tmp/sys/fs/cgroup/cpuacct
src/main/tools/linux-sandbox-pid1.cc:369: remount ro: /root/.cache/bazel/_bazel_root/f8087e59fd95af1ae29e8fcb7ff1a3dc/bazel-sandbox/143ece43-e9c8-4770-921e-152b0ee6e079-0/tmp/sys/fs/cgroup/memory
src/main/tools/linux-sandbox-pid1.cc:369: remount ro: /root/.cache/bazel/_bazel_root/f8087e59fd95af1ae29e8fcb7ff1a3dc/bazel-sandbox/143ece43-e9c8-4770-921e-152b0ee6e079-0/tmp/sys/fs/cgroup/devices
src/main/tools/linux-sandbox-pid1.cc:369: remount ro: /root/.cache/bazel/_bazel_root/f8087e59fd95af1ae29e8fcb7ff1a3dc/bazel-sandbox/143ece43-e9c8-4770-921e-152b0ee6e079-0/tmp/sys/fs/cgroup/freezer
src/main/tools/linux-sandbox-pid1.cc:369: remount ro: /root/.cache/bazel/_bazel_root/f8087e59fd95af1ae29e8fcb7ff1a3dc/bazel-sandbox/143ece43-e9c8-4770-921e-152b0ee6e079-0/tmp/sys/fs/cgroup/blkio
src/main/tools/linux-sandbox-pid1.cc:369: remount ro: /root/.cache/bazel/_bazel_root/f8087e59fd95af1ae29e8fcb7ff1a3dc/bazel-sandbox/143ece43-e9c8-4770-921e-152b0ee6e079-0/tmp/sys/fs/cgroup/perf_event
src/main/tools/linux-sandbox-pid1.cc:369: remount ro: /root/.cache/bazel/_bazel_root/f8087e59fd95af1ae29e8fcb7ff1a3dc/bazel-sandbox/143ece43-e9c8-4770-921e-152b0ee6e079-0/tmp/sys/fs/cgroup/hugetlb
src/main/tools/linux-sandbox-pid1.cc:369: remount ro: /root/.cache/bazel/_bazel_root/f8087e59fd95af1ae29e8fcb7ff1a3dc/bazel-sandbox/143ece43-e9c8-4770-921e-152b0ee6e079-0/tmp/sys/fs/cgroup/systemd
src/main/tools/linux-sandbox-pid1.cc:369: remount ro: /root/.cache/bazel/_bazel_root/f8087e59fd95af1ae29e8fcb7ff1a3dc/bazel-sandbox/143ece43-e9c8-4770-921e-152b0ee6e079-0/tmp/src
src/main/tools/linux-sandbox-pid1.cc:369: remount ro: /root/.cache/bazel/_bazel_root/f8087e59fd95af1ae29e8fcb7ff1a3dc/bazel-sandbox/143ece43-e9c8-4770-921e-152b0ee6e079-0/tmp/root/.cache/bazel
src/main/tools/linux-sandbox-pid1.cc:369: remount ro: /root/.cache/bazel/_bazel_root/f8087e59fd95af1ae29e8fcb7ff1a3dc/bazel-sandbox/143ece43-e9c8-4770-921e-152b0ee6e079-0/tmp/etc/resolv.conf
src/main/tools/linux-sandbox-pid1.cc:369: remount ro: /root/.cache/bazel/_bazel_root/f8087e59fd95af1ae29e8fcb7ff1a3dc/bazel-sandbox/143ece43-e9c8-4770-921e-152b0ee6e079-0/tmp/etc/hostname
src/main/tools/linux-sandbox-pid1.cc:369: remount ro: /root/.cache/bazel/_bazel_root/f8087e59fd95af1ae29e8fcb7ff1a3dc/bazel-sandbox/143ece43-e9c8-4770-921e-152b0ee6e079-0/tmp/etc/hosts
src/main/tools/linux-sandbox-pid1.cc:369: remount rw: /root/.cache/bazel/_bazel_root/f8087e59fd95af1ae29e8fcb7ff1a3dc/bazel-sandbox/143ece43-e9c8-4770-921e-152b0ee6e079-0/tmp/tmp
src/main/tools/linux-sandbox-pid1.cc:369: remount rw: /root/.cache/bazel/_bazel_root/f8087e59fd95af1ae29e8fcb7ff1a3dc/bazel-sandbox/143ece43-e9c8-4770-921e-152b0ee6e079-0/tmp/root/.cache/bazel/_bazel_root/f8087e59fd95af1ae29e8fcb7ff1a3dc/bazel-sandbox/143ece43-e9c8-4770-921e-152b0ee6e079-0/execroot/src
src/main/tools/linux-sandbox-pid1.cc:477: sigaction(32, &sa, NULL) failed
src/main/tools/linux-sandbox-pid1.cc:477: sigaction(33, &sa, NULL) failed
nobody@sandbox:~/.cache/bazel/_bazel_root/f8087e59fd95af1ae29e8fcb7ff1a3dc/bazel-sandbox/143ece43-e9c8-4770-921e-152b0ee6e079-0/execroot/src$ id
uid=65534(nobody) gid=65534(nogroup) groups=65534(nogroup)
nobody@sandbox:~/.cache/bazel/_bazel_root/f8087e59fd95af1ae29e8fcb7ff1a3dc/bazel-sandbox/143ece43-e9c8-4770-921e-152b0ee6e079-0/execroot/src$ ls
bazel-out  external  pkg
nobody@sandbox:~/.cache/bazel/_bazel_root/f8087e59fd95af1ae29e8fcb7ff1a3dc/bazel-sandbox/143ece43-e9c8-4770-921e-152b0ee6e079-0/execroot/src$ cat pkg/util/sets/
byte.go    doc.go     empty.go   int.go     int64.go   string.go  
nobody@sandbox:~/.cache/bazel/_bazel_root/f8087e59fd95af1ae29e8fcb7ff1a3dc/bazel-sandbox/143ece43-e9c8-4770-921e-152b0ee6e079-0/execroot/src$ cat pkg/util/sets/
byte.go    doc.go     empty.go   int.go     int64.go   string.go  
nobody@sandbox:~/.cache/bazel/_bazel_root/f8087e59fd95af1ae29e8fcb7ff1a3dc/bazel-sandbox/143ece43-e9c8-4770-921e-152b0ee6e079-0/execroot/src$ cat pkg/util/sets/empty.go 
cat: pkg/util/sets/empty.go: Permission denied
nobody@sandbox:~/.cache/bazel/_bazel_root/f8087e59fd95af1ae29e8fcb7ff1a3dc/bazel-sandbox/143ece43-e9c8-4770-921e-152b0ee6e079-0/execroot/src$ ls -lrth pkg/util/sets/
total 24K
lrwxrwxrwx 1 nobody nogroup  97 Oct 14 01:53 int.go -> /root/.cache/bazel/_bazel_root/f8087e59fd95af1ae29e8fcb7ff1a3dc/execroot/src/pkg/util/sets/int.go
lrwxrwxrwx 1 nobody nogroup  99 Oct 14 01:53 empty.go -> /root/.cache/bazel/_bazel_root/f8087e59fd95af1ae29e8fcb7ff1a3dc/execroot/src/pkg/util/sets/empty.go
lrwxrwxrwx 1 nobody nogroup  98 Oct 14 01:53 byte.go -> /root/.cache/bazel/_bazel_root/f8087e59fd95af1ae29e8fcb7ff1a3dc/execroot/src/pkg/util/sets/byte.go
lrwxrwxrwx 1 nobody nogroup 100 Oct 14 01:53 string.go -> /root/.cache/bazel/_bazel_root/f8087e59fd95af1ae29e8fcb7ff1a3dc/execroot/src/pkg/util/sets/string.go
lrwxrwxrwx 1 nobody nogroup  99 Oct 14 01:53 int64.go -> /root/.cache/bazel/_bazel_root/f8087e59fd95af1ae29e8fcb7ff1a3dc/execroot/src/pkg/util/sets/int64.go
lrwxrwxrwx 1 nobody nogroup  97 Oct 14 01:53 doc.go -> /root/.cache/bazel/_bazel_root/f8087e59fd95af1ae29e8fcb7ff1a3dc/execroot/src/pkg/util/sets/doc.go

@ekuefler
Copy link
Contributor Author

@philwo - not sure if it's the same issue as this or not, but sandboxing started giving me trouble again after 0.3.2: #1921

@philwo
Copy link
Member

philwo commented Oct 17, 2016

@mikedanese Unfortunately, there's no warning at the moment, but the fact that you're seeing messages from linux-sandbox.cc when using --sandbox_debug shows that it is using the full-featured Linux sandbox.

From your debugging we can see that the sandbox can access the symlink for empty.go, but not the target of the link:

lrwxrwxrwx 1 nobody nogroup 99 Oct 14 01:53 empty.go -> /root/.cache/bazel/_bazel_root/f8087e59fd95af1ae29e8fcb7ff1a3dc/execroot/src/pkg/util/sets/empty.go

It would be interesting to find out which part of the target path results in the permission denied error. For example, can you still access /root from your sandboxed bash? How far do you get until permission denied shows up?

@sherkwast sherkwast mentioned this issue Sep 16, 2017
Closed
@cfreemoser cfreemoser mentioned this issue Apr 27, 2019
Closed
@dmivankov dmivankov mentioned this issue Sep 17, 2021
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@philwo philwo

Labels
category: sandboxing P4 This is either out of scope or we don't have bandwidth to review a PR. (No assignee) type: feature request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@dcreager @philwo @ekuefler @mikedanese @damienmg