bazel-remote: init at 2019-01-12

[uri-canva]

Motivation for this change

Adds bazel-remote. This the implementation for a bazel cache http backend the official docs refer to, though it's experimental: https://docs.bazel.build/versions/master/remote-caching.html#bazel-remote-cache. It's authored by a member of the Bazel team, even though it's not published under the bazelbuild org because it's not officially supported. There is no officially supported self-hostable caching backend though.

Things done

• Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
• Built on platform(s)
  • NixOS
  • macOS
  • other Linux distributions
• Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
• Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
• Tested execution of all binary files (usually in ./result/bin/)
• Determined the impact on package closure size (by running nix path-info -S before and after)
• Assured whether relevant documentation is up to date
• Fits CONTRIBUTING.md.

---

[uri-canva]

This can change the hash of the dependencies, I've tested pythonPackages.tensorflow and it still matches, but both bazel-deps and bazel-watcher don't build with the version of Bazel we have on master so I can't check.

[Profpatsch]

@GrahamcOfBorg build bazel-remote

[Profpatsch]

Build fails for linux and darwin (not sure why the status is “neutral”).

[uri-canva]

Getting the same issue in #51723 (comment) as well since it uses gazelle too, will continue looking at it.

[uri-canva]

Done. Applied the same fix as in #51723 (comment).

[Profpatsch]

@GrahamcOfBorg build bazel-remote

[Profpatsch]

Nope, doesn’t build (on Linux):

ERROR: /build/source/config/BUILD.bazel:3:1: no such package '@in_gopkg_yaml_v2//': failed to fetch in_gopkg_yaml_v2: 2019/01/19 10:24:31 unrecognized import path "gopkg.in/yaml.v2"
 and referenced by '//config:go_default_library'
ERROR: /build/source/BUILD.bazel:11:1: no such package '@com_github_abbot_go_http_auth//': failed to fetch com_github_abbot_go_http_auth: # cd .; git clone https://github.com/abbot/go-http-auth /build/output/external/com_github_abbot_go_http_auth
Cloning into '/build/output/external/com_github_abbot_go_http_auth'...
error: could not lock config file /build/output/external/com_github_abbot_go_http_auth/.git/config: No such file or directory
fatal: could not set 'remote.origin.fetch' to '+refs/heads/*:refs/remotes/origin/*'
2019/01/19 10:24:31 exit status 128
 and referenced by '//:go_default_library'
ERROR: /build/source/config/BUILD.bazel:3:1: no such package '@in_gopkg_yaml_v2//': failed to fetch in_gopkg_yaml_v2: 2019/01/19 10:24:31 unrecognized import path "gopkg.in/yaml.v2"
 and referenced by '//config:go_default_library'
ERROR: Evaluation of query "deps(//:bazel-remote)" failed: errors were encountered while computing transitive closure

Also I’m confused about the ofborg build error for linux, we don’t use oraclejdk for bazel.

[uri-canva]

Sorry I updated the ubuntu box so it didn't have sandboxing configured anymore. Could repro the failure with sandboxing enabled, will fix.

[uri-canva]

Can't figure out why it's failing. Is there a way to run the sandboxed build in an interactive way? NixOS/nix#903 seems to imply there isn't.

[Profpatsch]

On linux you could try dropping yourself into a nix shell with the network namespace unshared:

sudo unshare --user --net -- nix-shell -A bazel /path/to/nixpkgs

Attention, you are root inside that shell. If it needs to download dependencies, first go into the shell once without unsharing. --user makes it a user namespace.

[uri-canva]

Won't have time to debug why it's failing on Linux, but I changed the supported platforms to be only Darwin so we can still use it as it is now.

[uri-canva]

ping @Profpatsch

[Profpatsch]

Ah, sorry. Merged.