Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[regression 0.3.2] Cannot create symbolic link to previously built artifact #1966

Closed
davido opened this issue Oct 20, 2016 · 6 comments
Closed

[regression 0.3.2] Cannot create symbolic link to previously built artifact #1966

davido opened this issue Oct 20, 2016 · 6 comments

Comments

@davido
Copy link
Contributor

davido commented Oct 20, 2016 •
edited

After upgrading to 0.3.2, working build rule is starting to fail:

genrule2 is defined as following:

def genrule2(out, cmd, **kwargs):
  cmd = ' && '.join([
    'ROOT=$$PWD',
    'TMP=$$(mktemp -d)',
    '(' + cmd + ')',
  ])
  native.genrule(
    cmd = cmd,
    outs = [out],
    **kwargs)

The rule diffy_image_files_ln is failing after the upgrade:

  DIFFY = glob(['src/main/resources/com/google/gerrit/client/diffy*.png'])
  load('//tools/bzl:genrule2.bzl', 'genrule2')

  java_library(
    name = 'diffy_image_files',
    resources = DIFFY,
  )

  genrule2(
    name = 'diffy_image_files_ln',
    srcs = [':diffy_image_files'],
    cmd = 'ln -s $$ROOT/$(location :diffy_image_files) $@',
    out = 'diffy_images.jar',
  )

Creating symbolic link to java_library is failing now:

  $ bazel build //gerrit-gwtui-common:diffy_image_files
  INFO: Found 1 target...
  Target //gerrit-gwtui-common:diffy_image_files up-to-date:
    bazel-bin/gerrit-gwtui-common/libdiffy_image_files.jar
  INFO: Elapsed time: 0.143s, Critical Path: 0.05s

  $ bazel build //gerrit-gwtui-common:diffy_image_files_ln  --verbose_failures --subcommands
INFO: Found 1 target...
>>>>> # //gerrit-gwtui-common:diffy_image_files_ln [action 'Executing genrule //gerrit-gwtui-common:diffy_image_files_ln']
(cd /home/davido/.cache/bazel/_bazel_davido/27a001f4182820ef315d8d2d4f1edafe/execroot/gerrit && \
  exec env - \
    PATH=/u01/app/oracle/product/11.2.0/xe/bin:/home/davido/pgm/pt:/opt/apache-maven-3.0.5/bin:/u01/app/oracle/product/11.2.0/xe/bin:/home/davido/pgm/pt:/opt/apache-maven-3.0.5/bin:/home/davido/bin:/usr/local/bin:/usr/bin:/bin:/usr/bin/X11:/usr/games:/opt/kde3/bin:/usr/lib/mit/bin:/usr/lib/mit/sbin:/usr/lib/qt3/bin:/home/davido/pgm/go/bin:/bin:/home/davido/pgm/go/bin:/home/davido/pgm/gocode/bin \
  /bin/bash -c 'source external/bazel_tools/tools/genrule/genrule-setup.sh; ROOT=$PWD && TMP=$(mktemp -d) && (ln -s $ROOT/bazel-out/local-fastbuild/bin/gerrit-gwtui-common/libdiffy_image_files.jar bazel-out/local-fastbuild/genfiles/gerrit-gwtui-common/diffy_images.jar)')
ERROR: /home/davido/projects/gerrit/gerrit-gwtui-common/BUILD:51:1: declared output 'gerrit-gwtui-common/diffy_images.jar' is a dangling symbolic link.
ERROR: /home/davido/projects/gerrit/gerrit-gwtui-common/BUILD:51:1: not all outputs were created.
Target //gerrit-gwtui-common:diffy_image_files_ln failed to build
INFO: Elapsed time: 0.151s, Critical Path: 0.08s

In fact, checking the path reveals that sandboxing is involved:

$ cd ~/.cache/bazel/_bazel_davido/27a001f4182820ef315d8d2d4f1edafe/execroot/gerrit
davido@linux-ucwl:~/.cache/bazel/_bazel_davido/27a001f4182820ef315d8d2d4f1edafe/execroot/gerrit$ ls -all bazel-out/local-fastbuild/genfiles/gerrit-gwtui-common/diffy_images.jar
lrwxrwxrwx 1 davido users 216 Oct 20 08:55 bazel-out/local-fastbuild/genfiles/gerrit-gwtui-common/diffy_images.jar -> /home/davido/.cache/bazel/_bazel_davido/27a001f4182820ef315d8d2d4f1edafe/bazel-sandbox/bc1993cb-4fae-4531-9871-1ff4bf6ce8f8-0/execroot/gerrit/bazel-out/local-fastbuild/bin/gerrit-gwtui-common/libdiffy_image_files.jar

Environment: Linux.

Probably related to this announcement in the release notes?

New sandboxing implementation for Linux in which all actions run in a separate execroot that contains input files as symlinks back to the originals in the workspace.

To reproduce, clone Gerrit Code Review from https://gerrit.googlesource.com/gerrit and run:

  $ bazel build //gerrit-gwtui-common:diffy_image_files_ln
@davido
Copy link
Contributor Author

davido commented Oct 20, 2016

I've created a reproducer repository with two build rules to simplify investigation: [1]. Clone it and issue:

  bazel build :guava_ln --verbose_failures --subcommands

Working log on 0.3.1:

$ bazel version
Build label: 0.3.1

$ bazel build :guava_ln --verbose_failures --subcommands 
INFO: Found 1 target...
>>>>> # //:guava_ln [action 'Executing genrule //:guava_ln']
(cd /home/davido/.cache/bazel/_bazel_davido/1c05526fe8c8cba2b825c0ac3eea91e4/execroot/bazel_test && \
  exec env - \
    PATH=/u01/app/oracle/product/11.2.0/xe/bin:/home/davido/pgm/pt:/opt/apache-maven-3.0.5/bin:/u01/app/oracle/product/11.2.0/xe/bin:/home/davido/pgm/pt:/opt/apache-maven-3.0.5/bin:/home/davido/bin:/usr/local/bin:/usr/bin:/bin:/usr/bin/X11:/usr/games:/opt/kde3/bin:/usr/lib/mit/bin:/usr/lib/mit/sbin:/usr/lib/qt3/bin:/home/davido/pgm/go/bin:/bin:/home/davido/pgm/go/bin:/home/davido/pgm/gocode/bin \
  /bin/bash -c 'source external/bazel_tools/tools/genrule/genrule-setup.sh; ROOT=$PWD && ln -s $ROOT/bazel-out/local-fastbuild/bin/libguava.jar bazel-out/local-fastbuild/genfiles/guava.zip')
Target //:guava_ln up-to-date:
  bazel-genfiles/guava.zip
INFO: Elapsed time: 3.644s, Critical Path: 0.07s

davido@linux-ucwl:~/projects/bazel_test (master %=)$ ls -all bazel-out/local-fastbuild/genfiles/guava.zip
lrwxrwxrwx 1 davido users 135 Oct 20 21:28 bazel-out/local-fastbuild/genfiles/guava.zip -> /home/davido/.cache/bazel/_bazel_davido/1c05526fe8c8cba2b825c0ac3eea91e4/execroot/bazel_test/bazel-out/local-fastbuild/bin/libguava.jar

Broken log on 0.3.2:

Build label: 0.3.2

$ bazel build :guava_ln --verbose_failures --subcommands 
INFO: Found 1 target...
>>>>> # //:guava_ln [action 'Executing genrule //:guava_ln']
(cd /home/davido/.cache/bazel/_bazel_davido/1c05526fe8c8cba2b825c0ac3eea91e4/execroot/bazel_test && \
  exec env - \
    PATH=/u01/app/oracle/product/11.2.0/xe/bin:/home/davido/pgm/pt:/opt/apache-maven-3.0.5/bin:/u01/app/oracle/product/11.2.0/xe/bin:/home/davido/pgm/pt:/opt/apache-maven-3.0.5/bin:/home/davido/bin:/usr/local/bin:/usr/bin:/bin:/usr/bin/X11:/usr/games:/opt/kde3/bin:/usr/lib/mit/bin:/usr/lib/mit/sbin:/usr/lib/qt3/bin:/home/davido/pgm/go/bin:/bin:/home/davido/pgm/go/bin:/home/davido/pgm/gocode/bin \
  /bin/bash -c 'source external/bazel_tools/tools/genrule/genrule-setup.sh; ROOT=$PWD && ln -s $ROOT/bazel-out/local-fastbuild/bin/libguava.jar bazel-out/local-fastbuild/genfiles/guava.zip')
ERROR: /home/davido/projects/bazel_test/BUILD:1:1: declared output 'guava.zip' is a dangling symbolic link.
ERROR: /home/davido/projects/bazel_test/BUILD:1:1: not all outputs were created.
Target //:guava_ln failed to build
INFO: Elapsed time: 3.156s, Critical Path: 0.08s

davido@linux-ucwl:~/projects/bazel_test (master %=)$ ls -all bazel-out/local-fastbuild/genfiles/guava.zip
lrwxrwxrwx 1 davido users 188 Oct 20 21:31 bazel-out/local-fastbuild/genfiles/guava.zip -> /home/davido/.cache/bazel/_bazel_davido/1c05526fe8c8cba2b825c0ac3eea91e4/bazel-sandbox/f28a81d2-85db-4e7d-84e0-e4b8d91397d5-0/execroot/bazel_test/bazel-out/local-fastbuild/bin/libguava.jar

@davido davido mentioned this issue Oct 20, 2016
Closed
@davido davido changed the title path issue after upgrade to 0.3.2 [regression 0.3.2] Cannot create symbolic link to previously built artifact Oct 23, 2016
@davido
Copy link
Contributor Author

davido commented Oct 23, 2016 •
edited

It turns out, that the breakage can be avoided, if --sandbox_debug option is passed to the build command. It seems that in this case sandbox directory tree is not deleted:

Without --sandbox_debug:

$ bazel build :guava_ln 
........
INFO: Found 1 target...
ERROR: /home/davido/projects/bazel_test/BUILD:1:1: declared output 'guava.zip' is a dangling symbolic link.
ERROR: /home/davido/projects/bazel_test/BUILD:1:1: not all outputs were created.
Target //:guava_ln failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 4.215s, Critical Path: 0.39s

With --sandbox_debug:

$ bazel build --sandbox_debug :guava_ln 
INFO: Found 1 target...
INFO: From Executing genrule //:guava_ln:
src/main/tools/linux-sandbox.cc:183: linux-sandbox-pid1 has PID 5746
src/main/tools/linux-sandbox-pid1.cc:241: tmpfs: /tmp
src/main/tools/linux-sandbox-pid1.cc:251: working dir: /home/davido/.cache/bazel/_bazel_davido/1c05526fe8c8cba2b825c0ac3eea91e4/bazel-sandbox/c6bd6cfb-27d6-4607-b096-9c63ca14b0d3-0/execroot/bazel_test
src/main/tools/linux-sandbox-pid1.cc:181: CreateTarget(home/davido/.cache/bazel/_bazel_davido/1c05526fe8c8cba2b825c0ac3eea91e4/bazel-sandbox/c6bd6cfb-27d6-4607-b096-9c63ca14b0d3-0/execroot/bazel_test, true)
src/main/tools/linux-sandbox-pid1.cc:369: remount ro: /home/davido/.cache/bazel/_bazel_davido/1c05526fe8c8cba2b825c0ac3eea91e4/bazel-sandbox/c6bd6cfb-27d6-4607-b096-9c63ca14b0d3-0/tmp
src/main/tools/linux-sandbox-pid1.cc:369: remount ro: /home/davido/.cache/bazel/_bazel_davido/1c05526fe8c8cba2b825c0ac3eea91e4/bazel-sandbox/c6bd6cfb-27d6-4607-b096-9c63ca14b0d3-0/tmp/dev
src/main/tools/linux-sandbox-pid1.cc:369: remount ro: /home/davido/.cache/bazel/_bazel_davido/1c05526fe8c8cba2b825c0ac3eea91e4/bazel-sandbox/c6bd6cfb-27d6-4607-b096-9c63ca14b0d3-0/tmp/dev/shm
src/main/tools/linux-sandbox-pid1.cc:369: remount ro: /home/davido/.cache/bazel/_bazel_davido/1c05526fe8c8cba2b825c0ac3eea91e4/bazel-sandbox/c6bd6cfb-27d6-4607-b096-9c63ca14b0d3-0/tmp/dev/pts
src/main/tools/linux-sandbox-pid1.cc:369: remount ro: /home/davido/.cache/bazel/_bazel_davido/1c05526fe8c8cba2b825c0ac3eea91e4/bazel-sandbox/c6bd6cfb-27d6-4607-b096-9c63ca14b0d3-0/tmp/dev/mqueue
src/main/tools/linux-sandbox-pid1.cc:369: remount ro: /home/davido/.cache/bazel/_bazel_davido/1c05526fe8c8cba2b825c0ac3eea91e4/bazel-sandbox/c6bd6cfb-27d6-4607-b096-9c63ca14b0d3-0/tmp/dev/hugepages
src/main/tools/linux-sandbox-pid1.cc:369: remount ro: /home/davido/.cache/bazel/_bazel_davido/1c05526fe8c8cba2b825c0ac3eea91e4/bazel-sandbox/c6bd6cfb-27d6-4607-b096-9c63ca14b0d3-0/tmp/proc
src/main/tools/linux-sandbox-pid1.cc:369: remount ro: /home/davido/.cache/bazel/_bazel_davido/1c05526fe8c8cba2b825c0ac3eea91e4/bazel-sandbox/c6bd6cfb-27d6-4607-b096-9c63ca14b0d3-0/tmp/proc/sys/fs/binfmt_misc
src/main/tools/linux-sandbox-pid1.cc:369: remount ro: /home/davido/.cache/bazel/_bazel_davido/1c05526fe8c8cba2b825c0ac3eea91e4/bazel-sandbox/c6bd6cfb-27d6-4607-b096-9c63ca14b0d3-0/tmp/sys
src/main/tools/linux-sandbox-pid1.cc:369: remount ro: /home/davido/.cache/bazel/_bazel_davido/1c05526fe8c8cba2b825c0ac3eea91e4/bazel-sandbox/c6bd6cfb-27d6-4607-b096-9c63ca14b0d3-0/tmp/sys/kernel/security
src/main/tools/linux-sandbox-pid1.cc:369: remount ro: /home/davido/.cache/bazel/_bazel_davido/1c05526fe8c8cba2b825c0ac3eea91e4/bazel-sandbox/c6bd6cfb-27d6-4607-b096-9c63ca14b0d3-0/tmp/sys/fs/cgroup
src/main/tools/linux-sandbox-pid1.cc:369: remount ro: /home/davido/.cache/bazel/_bazel_davido/1c05526fe8c8cba2b825c0ac3eea91e4/bazel-sandbox/c6bd6cfb-27d6-4607-b096-9c63ca14b0d3-0/tmp/sys/fs/cgroup/systemd
src/main/tools/linux-sandbox-pid1.cc:369: remount ro: /home/davido/.cache/bazel/_bazel_davido/1c05526fe8c8cba2b825c0ac3eea91e4/bazel-sandbox/c6bd6cfb-27d6-4607-b096-9c63ca14b0d3-0/tmp/sys/fs/cgroup/cpuset
src/main/tools/linux-sandbox-pid1.cc:369: remount ro: /home/davido/.cache/bazel/_bazel_davido/1c05526fe8c8cba2b825c0ac3eea91e4/bazel-sandbox/c6bd6cfb-27d6-4607-b096-9c63ca14b0d3-0/tmp/sys/fs/cgroup/cpu,cpuacct
src/main/tools/linux-sandbox-pid1.cc:369: remount ro: /home/davido/.cache/bazel/_bazel_davido/1c05526fe8c8cba2b825c0ac3eea91e4/bazel-sandbox/c6bd6cfb-27d6-4607-b096-9c63ca14b0d3-0/tmp/sys/fs/cgroup/blkio
src/main/tools/linux-sandbox-pid1.cc:369: remount ro: /home/davido/.cache/bazel/_bazel_davido/1c05526fe8c8cba2b825c0ac3eea91e4/bazel-sandbox/c6bd6cfb-27d6-4607-b096-9c63ca14b0d3-0/tmp/sys/fs/cgroup/memory
src/main/tools/linux-sandbox-pid1.cc:369: remount ro: /home/davido/.cache/bazel/_bazel_davido/1c05526fe8c8cba2b825c0ac3eea91e4/bazel-sandbox/c6bd6cfb-27d6-4607-b096-9c63ca14b0d3-0/tmp/sys/fs/cgroup/devices
src/main/tools/linux-sandbox-pid1.cc:369: remount ro: /home/davido/.cache/bazel/_bazel_davido/1c05526fe8c8cba2b825c0ac3eea91e4/bazel-sandbox/c6bd6cfb-27d6-4607-b096-9c63ca14b0d3-0/tmp/sys/fs/cgroup/freezer
src/main/tools/linux-sandbox-pid1.cc:369: remount ro: /home/davido/.cache/bazel/_bazel_davido/1c05526fe8c8cba2b825c0ac3eea91e4/bazel-sandbox/c6bd6cfb-27d6-4607-b096-9c63ca14b0d3-0/tmp/sys/fs/cgroup/net_cls,net_prio
src/main/tools/linux-sandbox-pid1.cc:369: remount ro: /home/davido/.cache/bazel/_bazel_davido/1c05526fe8c8cba2b825c0ac3eea91e4/bazel-sandbox/c6bd6cfb-27d6-4607-b096-9c63ca14b0d3-0/tmp/sys/fs/cgroup/perf_event
src/main/tools/linux-sandbox-pid1.cc:369: remount ro: /home/davido/.cache/bazel/_bazel_davido/1c05526fe8c8cba2b825c0ac3eea91e4/bazel-sandbox/c6bd6cfb-27d6-4607-b096-9c63ca14b0d3-0/tmp/sys/fs/cgroup/hugetlb
src/main/tools/linux-sandbox-pid1.cc:369: remount ro: /home/davido/.cache/bazel/_bazel_davido/1c05526fe8c8cba2b825c0ac3eea91e4/bazel-sandbox/c6bd6cfb-27d6-4607-b096-9c63ca14b0d3-0/tmp/sys/fs/pstore
src/main/tools/linux-sandbox-pid1.cc:369: remount ro: /home/davido/.cache/bazel/_bazel_davido/1c05526fe8c8cba2b825c0ac3eea91e4/bazel-sandbox/c6bd6cfb-27d6-4607-b096-9c63ca14b0d3-0/tmp/sys/kernel/debug
src/main/tools/linux-sandbox-pid1.cc:369: remount ro: /home/davido/.cache/bazel/_bazel_davido/1c05526fe8c8cba2b825c0ac3eea91e4/bazel-sandbox/c6bd6cfb-27d6-4607-b096-9c63ca14b0d3-0/tmp/sys/kernel/debug/tracing
src/main/tools/linux-sandbox-pid1.cc:369: remount ro: /home/davido/.cache/bazel/_bazel_davido/1c05526fe8c8cba2b825c0ac3eea91e4/bazel-sandbox/c6bd6cfb-27d6-4607-b096-9c63ca14b0d3-0/tmp/sys/fs/fuse/connections
src/main/tools/linux-sandbox-pid1.cc:369: remount ro: /home/davido/.cache/bazel/_bazel_davido/1c05526fe8c8cba2b825c0ac3eea91e4/bazel-sandbox/c6bd6cfb-27d6-4607-b096-9c63ca14b0d3-0/tmp/run
src/main/tools/linux-sandbox-pid1.cc:369: remount ro: /home/davido/.cache/bazel/_bazel_davido/1c05526fe8c8cba2b825c0ac3eea91e4/bazel-sandbox/c6bd6cfb-27d6-4607-b096-9c63ca14b0d3-0/tmp/run/user/1000/gvfs
src/main/tools/linux-sandbox-pid1.cc:369: remount rw: /home/davido/.cache/bazel/_bazel_davido/1c05526fe8c8cba2b825c0ac3eea91e4/bazel-sandbox/c6bd6cfb-27d6-4607-b096-9c63ca14b0d3-0/tmp/tmp
src/main/tools/linux-sandbox-pid1.cc:369: remount rw: /home/davido/.cache/bazel/_bazel_davido/1c05526fe8c8cba2b825c0ac3eea91e4/bazel-sandbox/c6bd6cfb-27d6-4607-b096-9c63ca14b0d3-0/tmp/home/davido/.cache/bazel/_bazel_davido/1c05526fe8c8cba2b825c0ac3eea91e4/bazel-sandbox/c6bd6cfb-27d6-4607-b096-9c63ca14b0d3-0/execroot/bazel_test
src/main/tools/linux-sandbox-pid1.cc:477: sigaction(32, &sa, NULL) failed
src/main/tools/linux-sandbox-pid1.cc:477: sigaction(33, &sa, NULL) failed
src/main/tools/linux-sandbox-pid1.cc:587: waitpid returned 2
src/main/tools/linux-sandbox-pid1.cc:607: child exited with code 0
src/main/tools/linux-sandbox.cc:223: child exited normally with exitcode 0
Target //:guava_ln up-to-date:
  bazel-genfiles/guava.zip
INFO: Elapsed time: 0.159s, Critical Path: 0.05s

Let's check, where guava.zip is pointing too:

$ ls -all bazel-genfiles/guava.zip
lrwxrwxrwx 1 davido users 188 Oct 23 19:57 bazel-genfiles/guava.zip -> /home/davido/.cache/bazel/_bazel_davido/1c05526fe8c8cba2b825c0ac3eea91e4/bazel-sandbox/c6bd6cfb-27d6-4607-b096-9c63ca14b0d3-0/execroot/bazel_test/bazel-out/local-fastbuild/bin/libguava.jar

What was Bazel 0.3.1 doing?

$ bazel version
Build label: 0.3.1

$ bazel build :guava_ln
INFO: Found 1 target...
Target //:guava_ln up-to-date:
  bazel-genfiles/guava.zip
INFO: Elapsed time: 5.427s, Critical Path: 0.76s

Let's check, where guava.zip was pointing too in 0.3.1:

$ ls -all bazel-genfiles/guava.zip
lrwxrwxrwx 1 davido users 135 Oct 23 20:21 bazel-genfiles/guava.zip -> /home/davido/.cache/bazel/_bazel_davido/1c05526fe8c8cba2b825c0ac3eea91e4/execroot/bazel_test/bazel-out/local-fastbuild/bin/libguava.jar

No sandboxing was involved.

@hanwen
Copy link
Contributor

hanwen commented Oct 24, 2016

why are you creating symlinks pointing to absolute paths? They aren't hermetic because they encode data from the workspace absolute paths.

@davido
Copy link
Contributor Author

davido commented Oct 24, 2016

Well, that was the way, it was done once in Buck and ported to Bazel 0.3.1 and it worked there.

@hanwen
Copy link
Contributor

hanwen commented Oct 24, 2016

Is the value of $PWD a documented part of the API? If not, this is probably not a regression.

@davido
Copy link
Contributor Author

davido commented Oct 24, 2016

We've solved the problem by not using symbolic link.

@davido davido closed this as completed Oct 24, 2016
lucamilanesio pushed a commit to GerritCodeReview/gerrit that referenced this issue Oct 24, 2016
@davido
Bazel: Simplify diffy_logo rules
0e12113 
There is a regression in Bazel 0.3.2 release: [1] that prevents creating
a symbolic link to previously built artifact. In this case we don't
really need to do that, as we can depend on previously built library and
do not need to create a zip from java_library and import it again as
java_library.

[1] bazelbuild/bazel#1966

Change-Id: I16a62e2fbb3e2d1de377eeef70061298bb2bb745
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@hanwen @davido