Compilation error in a container

[comes5]

I am trying to build incubator-heron master in a container with Ubuntu, I managed to install everything but bazel when building the heron folder gives me an error, probably related to Python.
Can someone help me? I have attached all the output from the build comand, specifically bazel build --config=ubuntu heron/...

output.log

[kramasamy]

@comes5 - can you include what error you are getting?

[kramasamy]

Alternatively you could use the docker image at

streamlio/build-ubuntu14.0:latest 

it has everything installed already.

[comes5]

The error I am mentioning is in the log file I posted:

ERROR: /incubator-heron/heron/tools/admin/src/python/BUILD:22:1: PexPython heron/tools/admin/src/python/heron-admin.pex failed (Exit 1) **** Failed to install netifaces-0.10.6 (caused by: NonZeroExit("received exit code 1 during execution of ['/usr/bin/python2.7', '-', 'bdist_wheel', '--dist-dir=/tmp/tmpwn7Syj']while trying to execute['/usr/bin/python2.7', '-', 'bdist_wheel', '--dist-dir=/tmp/tmpwn7Syj']",) ):

The problem is that I want to use Dhalion to implement new policies for Heron so I need to modify the source code and doing some tests with it.

[tomncooper]

Can you clarify if you are using the Docker containers defined in the Heron repo (docker/compile) or you are constructing your own?

[comes5]

Just constructing it on my own using the ubuntu default container.
Here is my dockerfile:

Dockerfile.txt

[tomncooper]

I would recommend using the dockerfiles in the docker/compile directory they are used routinely for compilation so should work. Can you give the instructions in the docker/Readme.md a try?

[comes5]

Ok tried to compile using the instructions in /docker/Readme.md but there seem to be another error:
INFO: Starting clean (this may take a while). Consider using --async if the clean takes more than several minutes. Creating packages INFO: Analysed target //scripts/packages:tarpkgs (240 packages loaded). INFO: Found 1 target... ERROR: /scratch/heron/metricsmgr/src/thrift/BUILD:3:1: error executing shell command: 'set -e rm -rf bazel-out/k8-opt/genfiles/heron/metricsmgr/src/thrift/thrift_scribe_java_src.srcjar.srcs mkdir bazel-out/k8-opt/genfiles/heron/metricsmgr/src/thrift/thrift_scribe_java_src.srcjar.srcs...' failed (Segmentation fault): bash failed: error executing command (cd /root/.cache/bazel/_bazel_root/c481f31d0aff7f9fd86654fb84c9a629/execroot/org_apache_heron && \ exec env - \ PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin \ /bin/bash -c 'set -e rm -rf bazel-out/k8-opt/genfiles/heron/metricsmgr/src/thrift/thrift_scribe_java_src.srcjar.srcs mkdir bazel-out/k8-opt/genfiles/heron/metricsmgr/src/thrift/thrift_scribe_java_src.srcjar.srcs third_party/thrift/thrift-linux-x86_64.exe -r --gen java -o bazel-out/k8-opt/genfiles/heron/metricsmgr/src/thrift/thrift_scribe_java_src.srcjar.srcs heron/metricsmgr/src/thrift/scribe.thrift jar cMf bazel-out/k8-opt/genfiles/heron/metricsmgr/src/thrift/thrift_scribe_java_src.srcjar -C bazel-out/k8-opt/genfiles/heron/metricsmgr/src/thrift/thrift_scribe_java_src.srcjar.srcs . rm -rf bazel-out/k8-opt/genfiles/heron/metricsmgr/src/thrift/thrift_scribe_java_src.srcjar.srcs'): bash failed: error executing command (cd /root/.cache/bazel/_bazel_root/c481f31d0aff7f9fd86654fb84c9a629/execroot/org_apache_heron && \ exec env - \ PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin \ /bin/bash -c 'set -e rm -rf bazel-out/k8-opt/genfiles/heron/metricsmgr/src/thrift/thrift_scribe_java_src.srcjar.srcs mkdir bazel-out/k8-opt/genfiles/heron/metricsmgr/src/thrift/thrift_scribe_java_src.srcjar.srcs third_party/thrift/thrift-linux-x86_64.exe -r --gen java -o bazel-out/k8-opt/genfiles/heron/metricsmgr/src/thrift/thrift_scribe_java_src.srcjar.srcs heron/metricsmgr/src/thrift/scribe.thrift jar cMf bazel-out/k8-opt/genfiles/heron/metricsmgr/src/thrift/thrift_scribe_java_src.srcjar -C bazel-out/k8-opt/genfiles/heron/metricsmgr/src/thrift/thrift_scribe_java_src.srcjar.srcs . rm -rf bazel-out/k8-opt/genfiles/heron/metricsmgr/src/thrift/thrift_scribe_java_src.srcjar.srcs') /bin/bash: line 3: 2632 Segmentation fault (core dumped) third_party/thrift/thrift-linux-x86_64.exe -r --gen java -o bazel-out/k8-opt/genfiles/heron/metricsmgr/src/thrift/thrift_scribe_java_src.srcjar.srcs heron/metricsmgr/src/thrift/scribe.thrift Target //scripts/packages:tarpkgs failed to build INFO: Elapsed time: 168.796s, Critical Path: 115.46s FAILED: Build did NOT complete successfully Cleaning up scratch dir

[kramasamy]

@comes5 - in the new PR #2879 - we are removing the scribe and the thrift executable. Once this is merged, this error will go away.

[comes5]

So, I am trying with virtualbox now, following the heron documentation with ubuntu 16.04. So far no errors from thrift and scribe, but the build fails after a long time building:
ERROR: Process exited with status 1: Process exited with status 1 fatal: Not a git repository (or any of the parent directories): .git Failed to run command to check head: git rev-parse --abbrev-ref HEAD INFO: Elapsed time: 1525.185s, Critical Path: 69.10s INFO: 1914 processes: 1603 local, 311 worker. FAILED: Build did NOT complete successfully

[tomncooper]

Have you checked out one of the release tags or are you building from the head of the master branch? Try checking out the latest tag, 0.17.8, and compiling from there?

[comes5]

Solved, Bazel couldn't find the .git directory which I had removed. One question though: do I have to build everything if I modify only one component and I want to test it?

[nlu90]

@comes5 In short, Yes. You'll need to build the whole project to get the core jars for testing. But since bazel is smart enough to do the incremental building, it will only build targets affected by your changes after the initial full building.