Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

buildBazelPackage: cleanup hacks and enforce them for all builds #64904

Merged
merged 1 commit into from Jul 17, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
43 changes: 13 additions & 30 deletions pkgs/build-support/build-bazel-package/default.nix
@@ -1,4 +1,4 @@
{ stdenv, bazel, cacert, enableNixHacks ? true }:
{ stdenv, bazel, cacert }:

args@{ name, bazelFlags ? [], bazelTarget, buildAttrs, fetchAttrs, ... }:

Expand Down Expand Up @@ -37,7 +37,15 @@ in stdenv.mkDerivation (fBuildAttrs // {

# We disable multithreading for the fetching phase since it can lead to timeouts with many dependencies/threads:
# https://github.com/bazelbuild/bazel/issues/6502
BAZEL_USE_CPP_ONLY_TOOLCHAIN=1 USER=homeless-shelter bazel --output_base="$bazelOut" --output_user_root="$bazelUserRoot" fetch --loading_phase_threads=1 $bazelFlags $bazelTarget
BAZEL_USE_CPP_ONLY_TOOLCHAIN=1 \
USER=homeless-shelter \
bazel \
--output_base="$bazelOut" \
--output_user_root="$bazelUserRoot" \
fetch \
--loading_phase_threads=1 \
$bazelFlags \
$bazelTarget

runHook postBuild
'';
Expand Down Expand Up @@ -74,12 +82,14 @@ in stdenv.mkDerivation (fBuildAttrs // {
'';

dontFixup = true;
allowedRequisites = [];

outputHashMode = "recursive";
outputHashAlgo = "sha256";
outputHash = fetchAttrs.sha256;
});

nativeBuildInputs = fBuildAttrs.nativeBuildInputs or [] ++ [ (if enableNixHacks then (bazel.override { enableNixHacks = true; }) else bazel) ];
nativeBuildInputs = fBuildAttrs.nativeBuildInputs or [] ++ [ (bazel.override { enableNixHacks = true; }) ];

preHook = fBuildAttrs.preHook or "" + ''
export bazelOut="$NIX_BUILD_TOP/output"
Expand All @@ -99,40 +109,13 @@ in stdenv.mkDerivation (fBuildAttrs // {
buildPhase = fBuildAttrs.buildPhase or ''
runHook preBuild

# Bazel sandboxes the execution of the tools it invokes, so even though we are
# calling the correct nix wrappers, the values of the environment variables
# the wrappers are expecting will not be set. So instead of relying on the
# wrappers picking them up, pass them in explicitly via `--copt`, `--linkopt`
# and related flags.
#
copts=()
host_copts=()
for flag in $NIX_CFLAGS_COMPILE; do
copts+=( "--copt=$flag" )
host_copts+=( "--host_copt=$flag" )
done
for flag in $NIX_CXXSTDLIB_COMPILE; do
copts+=( "--copt=$flag" )
host_copts+=( "--host_copt=$flag" )
done
linkopts=()
host_linkopts=()
for flag in $NIX_LD_FLAGS; do
linkopts+=( "--linkopt=$flag" )
host_linkopts+=( "--host_linkopt=$flag" )
done

BAZEL_USE_CPP_ONLY_TOOLCHAIN=1 \
USER=homeless-shelter \
bazel \
--output_base="$bazelOut" \
--output_user_root="$bazelUserRoot" \
build \
-j $NIX_BUILD_CORES \
"''${copts[@]}" \
"''${host_copts[@]}" \
"''${linkopts[@]}" \
"''${host_linkopts[@]}" \
$bazelFlags \
$bazelTarget

Expand Down
28 changes: 0 additions & 28 deletions pkgs/development/tools/build-managers/bazel/nix-hacks.patch
@@ -1,31 +1,3 @@
diff -Naur a/src/main/java/com/google/devtools/build/lib/rules/repository/RepositoryDelegatorFunction.java b/src/main/java/com/google/devtools/build/lib/rules/repository/RepositoryDelegatorFunction.java
--- a/src/main/java/com/google/devtools/build/lib/rules/repository/RepositoryDelegatorFunction.java 2019-06-12 20:39:37.420705161 -0700
+++ b/src/main/java/com/google/devtools/build/lib/rules/repository/RepositoryDelegatorFunction.java 2019-06-12 20:44:18.894429744 -0700
@@ -428,24 +428,7 @@
try {
content = FileSystemUtils.readContent(markerPath, StandardCharsets.UTF_8);
String markerRuleKey = readMarkerFile(content, markerData);
- boolean verified = false;
- if (Preconditions.checkNotNull(ruleKey).equals(markerRuleKey)
- && Objects.equals(
- markerData.get(MANAGED_DIRECTORIES_MARKER),
- this.markerData.get(MANAGED_DIRECTORIES_MARKER))) {
- verified = handler.verifyMarkerData(rule, markerData, env);
- if (env.valuesMissing()) {
- return null;
- }
- }
-
- if (verified) {
return new Fingerprint().addString(content).digestAndReset();
- } else {
- // So that we are in a consistent state if something happens while fetching the repository
- markerPath.delete();
- return null;
- }
} catch (IOException e) {
throw new RepositoryFunctionException(e, Transience.TRANSIENT);
}
diff -Naur a/src/main/java/com/google/devtools/build/lib/shell/JavaSubprocessFactory.java b/src/main/java/com/google/devtools/build/lib/shell/JavaSubprocessFactory.java
--- a/src/main/java/com/google/devtools/build/lib/shell/JavaSubprocessFactory.java 2019-06-12 20:39:37.538708196 -0700
+++ b/src/main/java/com/google/devtools/build/lib/shell/JavaSubprocessFactory.java 2019-06-12 20:44:18.863429602 -0700
Expand Down
8 changes: 2 additions & 6 deletions pkgs/top-level/all-packages.nix
Expand Up @@ -8915,15 +8915,11 @@ in
buildozer = bazel-buildtools;
unused_deps = bazel-buildtools;

bazel-deps = callPackage ../development/tools/build-managers/bazel/bazel-deps {
buildBazelPackage = buildBazelPackage.override { enableNixHacks = false; };
};
bazel-deps = callPackage ../development/tools/build-managers/bazel/bazel-deps { };

bazel-remote = callPackage ../development/tools/build-managers/bazel/bazel-remote { };

bazel-watcher = callPackage ../development/tools/bazel-watcher {
buildBazelPackage = buildBazelPackage.override { enableNixHacks = false; };
};
bazel-watcher = callPackage ../development/tools/bazel-watcher { };

bazelisk = callPackage ../development/tools/bazelisk { };

Expand Down