no such package 'util/hash': BUILD file not found

[arrowd]

System information

• Have I written custom code (as opposed to using a stock example script provided in TensorFlow): No.
• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): FreeBSD 12.0-ALPHA3
• Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device:
• TensorFlow installed from (source or binary): source
• TensorFlow version (use command below): 1.11.0
• Python version: 2.7.15
• Bazel version (if compiling from source): 0.17.1
• GCC/Compiler version (if compiling from source): clang version 6.0.1
• CUDA/cuDNN version:
• GPU model and memory:
• Exact command to reproduce: bazel fetch --repository_cache=/path/to/empty/bazel_cache //tensorflow:libtensorflow.so

Describe the problem

I'm doing the port for tensorflow and trying to fetch everything that's required for build first. After extracting tensorflow tarball I run bazel fetch --repository_cache=/path/to/empty/bazel_cache //tensorflow:libtensorflow.so and get the following error:

ERROR: /wrkdirs/usr/ports/science/tensorflow-core/work/tensorflow-1.11.0/tensorflow/core/common_runtime/eager/BUILD:215:1: no such package 'util/hash': BUILD file not found on package path and referenced by '//tensorflow/core/common_runtime/eager:attr_builder'
ERROR: Evaluation of query "deps(//tensorflow:libtensorflow.so)" failed: errors were encountered while computing transitive closure

[wt-huang]

@arrowd Looks like some files are missing, try to install TensorFlow from the binary.

[arrowd]

I doubt there are binaries for FreeBSD. This is why I'm doing the port, after all. Can you point me to FreeBSD binaries?

[wt-huang]

@arrowd You are correct for FreeBSD it is better to install from port. Try this link to download the port for bazel 0.2.2b. You may want to use clang 3.8.

[arrowd]

I already have the latest bazel. The problem is not with bazel, but TensorFlow.

[wt-huang]

TensorFlow works on a score of platforms as shown in this link. You can also try to install TensorFlow on FreeBSD following this.

[arrowd]

I don't quite get what you are saying. I'm simply trying to compile latest TensorFlow release with latest Bazel release and https://www.tensorflow.org/install/source is exactly steps I'm following. Is this supported?

The manual you provided uses ancient Bazel to build old version of TF. I don't need it, I want to create a package of recent TF for FreeBSD users.

[arrowd]

Ping? Maybe I'm just doing something wrong? Is

bazel fetch --repository_cache=/path/to/empty/bazel_cache //tensorflow:libtensorflow.so

command supposed to work at all?

[acowley]

I'd like to add a confirmation that I have this issue, too. In NixOS we try to use bazel by fetching, patching, then building. I am using a built-from-source version of AMD's ROCm patches to tensorflow, but had to ultimately create a new bazel wrapper to apply the needed patches during the build since the fetch command fails with this problem.

[wt-huang]

@arrowd The instructions can be adapted to the newer TensorFlow version but it won't be trivial.
In the meantime, you may need to check whether all protobuf patches are applied from ports before fetching and building, after which you can build again. bazel fetch does work on most Linux systems.

@acowley glad you found a solution. Applying all needed patches is essential to get around this problem. Installing TensorFlow from source is usually challenging which is why we recommend binary.

[arrowd]

In the meantime, you may need to check whether all protobuf patches are applied from ports before fetching and building

Can you please elaborate on this? What protobuf patches?

[TravisWhitaker]

Was just hit with this as well.

@acowley I'm using Nix to build TF as well, how did you end up working around this? Did you override buildBazelPackage?

[TravisWhitaker]

FWIW versions 1.11, 1.10.1, and 1.10.0 are affected. Version 1.9.0 isn't, it seems.

[acowley]

@TravisWhitaker Yes, exactly, I used a custom buildBazelPackage. I abandoned bazel fetch due to this Issue, and baked the patches to the hard coded /usr/bin/ar paths used by bazel into src/main/tools/process-wrapper-legacy.cc (part of the bazel source) itself. This is horribly gross, but I also needed to patch dozens of other paths in tensorflow, so I could stomach it as a temporary fix.

rocm-bazel = (pkgs.bazel.override {enableNixHacks = true;}).overrideAttrs (old: {
    # Packages that bazel fetches often include hard-coded paths to
    # /usr/bin/ar. We in turn hard code a fix for that here. If
    # bazel's fetch command works for your build, then this is not
    # needed as these paths can be patched after download but before
    # build. If fetching does not work, and you need to rely upon
    # downloads during the build phase, this can help.
    postPatch = old.postPatch + ''
      find -type f -name CROSSTOOL\* -exec sed -i -e 's,/usr/bin/ar,${pkgs.binutils.bintools}/bin/ar,g' {} \;
      sed -e 's|if (execvp(opt.args\[0\]|if (execvp(strcmp(opt.args[0], "/usr/bin/ar") == 0 ? "${pkgs.binutils.bintools}/bin/ar" : opt.args[0]|' \
          -e 's|\(#include <vector>\)|\1\n#include <cstring>|' \
          -i src/main/tools/process-wrapper-legacy.cc
    '';
  });

[TravisWhitaker]

@acowley Ah nice, I'll give that a go. Thanks!

[wt-huang]

Thanks for sharing your fix for NixOS @acowley.

The patches that may be helpful for FreeBSD case can be found in https://svnweb.freebsd.org/ports/head/devel/protobuf/files/, locate the correct files that match with the FreeBSD version installed.

[jvishnuvardhan]

Closing this out since I understand it to be resolved, but please let me know if I'm mistaken. If the issue is persists again, please open a new ticket. Thanks!