Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Java/proto project fails #199

Closed
ah-quant opened this issue Nov 27, 2018 · 3 comments
Closed

Java/proto project fails #199

ah-quant opened this issue Nov 27, 2018 · 3 comments

Comments

@ah-quant
Copy link

I'm trying to get build-farm running on Ubuntu.

I successfully installed all required dependencies and Bazel 0.19.2, cloned the repo and started server and worker with the example configuration, each in its own screen.

This failed until I added a wrapper script exporting BAZEL_SH and JAVA_HOME.
Now both start up.

As demonstrated by the local build of build-farm itself, I can build java projects.

Starting it from a remote machine, I get loads of errors like this:

ERROR: /home/user/.cache/bazel/_bazel_user/f00d0a9338d336ba97aeac71037e0456/external/io_grpc_grpc_java/compiler/BUILD.bazel:1:1: Couldn't build file external/io_grpc_grpc_java/compiler/_objs/grpc_java_plugin/java_generator.o: undeclared inclusion(s) in rule '@io_grpc_grpc_java//compiler:grpc_java_plugin':
this rule is missing dependency declarations for the following files included by 'external/io_grpc_grpc_java/compiler/src/java_plugin/cpp/java_generator.cpp':
  '/usr/lib/gcc/x86_64-linux-gnu/5/include/stddef.h'
  '/usr/lib/gcc/x86_64-linux-gnu/5/include/stdarg.h'
  '/usr/lib/gcc/x86_64-linux-gnu/5/include/stdint.h'
  '/usr/lib/gcc/x86_64-linux-gnu/5/include/x86intrin.h'
  '/usr/lib/gcc/x86_64-linux-gnu/5/include/ia32intrin.h'
  '/usr/lib/gcc/x86_64-linux-gnu/5/include/mmintrin.h'
  '/usr/lib/gcc/x86_64-linux-gnu/5/include/xmmintrin.h'
  '/usr/lib/gcc/x86_64-linux-gnu/5/include/mm_malloc.h'
  '/usr/lib/gcc/x86_64-linux-gnu/5/include/emmintrin.h'
  '/usr/lib/gcc/x86_64-linux-gnu/5/include/pmmintrin.h'
  '/usr/lib/gcc/x86_64-linux-gnu/5/include/tmmintrin.h'
  '/usr/lib/gcc/x86_64-linux-gnu/5/include/ammintrin.h'
  '/usr/lib/gcc/x86_64-linux-gnu/5/include/smmintrin.h'
  '/usr/lib/gcc/x86_64-linux-gnu/5/include/popcntintrin.h'
  '/usr/lib/gcc/x86_64-linux-gnu/5/include/wmmintrin.h'
  '/usr/lib/gcc/x86_64-linux-gnu/5/include/immintrin.h'
  '/usr/lib/gcc/x86_64-linux-gnu/5/include/avxintrin.h'
  '/usr/lib/gcc/x86_64-linux-gnu/5/include/avx2intrin.h'
  '/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512fintrin.h'
  '/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512erintrin.h'
  '/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512pfintrin.h'
  '/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512cdintrin.h'
  '/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512vlintrin.h'
  '/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512bwintrin.h'
  '/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512dqintrin.h'
  '/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512vlbwintrin.h'
  '/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512vldqintrin.h'
  '/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512ifmaintrin.h'
  '/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512ifmavlintrin.h'
  '/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512vbmiintrin.h'
  '/usr/lib/gcc/x86_64-linux-gnu/5/include/avx512vbmivlintrin.h'
  '/usr/lib/gcc/x86_64-linux-gnu/5/include/shaintrin.h'
  '/usr/lib/gcc/x86_64-linux-gnu/5/include/lzcntintrin.h'
  '/usr/lib/gcc/x86_64-linux-gnu/5/include/bmiintrin.h'
  '/usr/lib/gcc/x86_64-linux-gnu/5/include/bmi2intrin.h'
  '/usr/lib/gcc/x86_64-linux-gnu/5/include/fmaintrin.h'
  '/usr/lib/gcc/x86_64-linux-gnu/5/include/f16cintrin.h'
  '/usr/lib/gcc/x86_64-linux-gnu/5/include/rtmintrin.h'
  '/usr/lib/gcc/x86_64-linux-gnu/5/include/xtestintrin.h'
  '/usr/lib/gcc/x86_64-linux-gnu/5/include/mm3dnow.h'
  '/usr/lib/gcc/x86_64-linux-gnu/5/include/prfchwintrin.h'
  '/usr/lib/gcc/x86_64-linux-gnu/5/include/fma4intrin.h'
  '/usr/lib/gcc/x86_64-linux-gnu/5/include/xopintrin.h'
  '/usr/lib/gcc/x86_64-linux-gnu/5/include/lwpintrin.h'
  '/usr/lib/gcc/x86_64-linux-gnu/5/include/tbmintrin.h'
  '/usr/lib/gcc/x86_64-linux-gnu/5/include/rdseedintrin.h'
  '/usr/lib/gcc/x86_64-linux-gnu/5/include/fxsrintrin.h'
  '/usr/lib/gcc/x86_64-linux-gnu/5/include/xsaveintrin.h'
  '/usr/lib/gcc/x86_64-linux-gnu/5/include/xsaveoptintrin.h'
  '/usr/lib/gcc/x86_64-linux-gnu/5/include/adxintrin.h'
  '/usr/lib/gcc/x86_64-linux-gnu/5/include/clwbintrin.h'
  '/usr/lib/gcc/x86_64-linux-gnu/5/include/pcommitintrin.h'
  '/usr/lib/gcc/x86_64-linux-gnu/5/include/clflushoptintrin.h'
  '/usr/lib/gcc/x86_64-linux-gnu/5/include/xsavesintrin.h'
  '/usr/lib/gcc/x86_64-linux-gnu/5/include/xsavecintrin.h'
  '/usr/lib/gcc/x86_64-linux-gnu/5/include/mwaitxintrin.h'
  '/usr/lib/gcc/x86_64-linux-gnu/5/include-fixed/limits.h'
  '/usr/lib/gcc/x86_64-linux-gnu/5/include-fixed/syslimits.h'

With logging enabled, all looks fine to me on both server and worker:

The server has lots of log lines like these:

FEIN: default_memory_instance::execute(2d1874ae7b2bf4f84fde94b97fc8e6f5ceeaf1f5751b093d6d2fa6b8a69f5108/142): default_memory_instance/operations/f72047dd-cefe-425e-9d0d-936ffa45c870 [Di Nov 27 17:14:23 MEZ 2018]

The worker log lines come in blocks and look like this:

FEIN: InputFetchStage::iterate(default_memory_instance/operations/4ba6f2f0-efca-4a38-aec2-33c7ebd00b03): Starting [Di Nov 27 17:14:37 MEZ 2018]
FEIN: Executor::executeCommand(default_memory_instance/operations/f72047dd-cefe-425e-9d0d-936ffa45c870): Completed command: exit code 0 [Di Nov 27 17:14:37 MEZ 2018]
FEIN: ExecuteActionStage::iterate(default_memory_instance/operations/f72047dd-cefe-425e-9d0d-936ffa45c870): 5,23900ms (0,0260000ms stalled) exit code: 0, 0/1 [Di Nov 27 17:14:37 MEZ 2018]
FEIN: ReportResultStage::iterate(default_memory_instance/operations/5c18d9b3-654d-47cd-90f2-1991813126f9): 6,18000ms (0,00300000ms stalled) Success [Di Nov 27 17:14:37 MEZ 2018]

I'm at a loss and I'd really love to get this running. If you need more info, I'll give my best to assist.
@RNabel
Copy link

RNabel commented Nov 27, 2018

(This is a bit of a drive-by since I haven't used buildfarm)
I got this error when experimenting with docker-sandboxing yesterday, and found that the CROSSTOOL added cxx_builtin_include_directory for a gcc/clang version different from the container version.
Example from the bazel-toolchains repo: https://github.com/bazelbuild/bazel-toolchains/blob/master/configs/ubuntu16_04_clang/1.0/bazel_0.14.1/default/CROSSTOOL#L152
In my case, I was able to "just" add/remove lines that point to the headers of the correct version.

@werkt
Copy link
Collaborator

werkt commented Nov 27, 2018

This is technically an issue with bazel itself, and one that I've seen crop up in a build of bazel that crossed ubuntu version boundaries. As @RNabel is indicating, your client is perceiving the gcc 5 environment that your workers are using during compilation as depending upon headers that bazel does not believe are enumerated as inputs. Getting these two environments matched will be the bulk of the work required to make this work properly.

@ah-quant
Copy link
Author

Thank you @RNabel and @werkt - upgrading the host OS for the workers indeed solved my problem.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@werkt @RNabel @ah-quant