Compiling bazel natively on armv7l #606
Comments
|
You can't use protoc and libprotobuf with different versions, and you may On Sun, Nov 15, 2015 at 9:31 PM, Víctor Mayoral Vilches <
|
damienmg
added
type: feature request
P3
|
Thanks @ulfjack. I've removed https://gist.github.com/vmayoral/ae066b7c87614273a7d8 I'm either missing something or I've just bumped into a new use case. @damienmg, i've seen you tagged this issue with I've hacked I'd appreciate some help here. Either the 2.x or the 3.0.0 way. |
|
I've gone ahead and reproduced it in a clean armhf Still getting the result described above. |
|
So, I checked the code after I posted and we're at a pre-release version of 3.0.0, if I understand correctly. It's possible that there were further changes to protobuf that are incompatible with the version we have checked in. I don't see anything in the protobuf release notes that looks related though. You could try: https://github.com/google/protobuf/releases/tag/v3.0.0-alpha-3 I think that's the version we're using. |
|
There is a beta release of Protobuf 3 out: https://github.com/google/protobuf/releases/tag/v3.0.0-beta-1 Would it be worthwhile for us to upgrade to 3.0.0-beta-1 or should we wait? |
|
@ulfjack, @davidzchen thanks. I've just reproduced it once again in a Ubuntu 14.04 ARM machine in the cloud (Scaleway) and just got a similar result https://gist.github.com/vmayoral/4e382cfcfa10e0a579bc so I'll just conclude that "as it is", protobuf master is incompatible. I'll probably put some time tonight and retry with the pointers you provide. |
|
Confirmed on arm64 platform on https://www.runabove.com/armcloud.xml |
vmayoral
changed the title
|
Thanks @shaygalon, The thing is that while on On my Not sure if this is useful. But I've been observing it from the beginning. |
|
Failed while using |
|
Something interesting to note is that while on Above I mentioned that hacking these |
|
Got that part working with protobuf 3.0.0-beta1 on arm64. Rebuilt protobuf and modified scripts to use arm64 binary, as well as use the new jar file from protobuf. However, it then complains about no toolchain for cpu 'unknown'. Guess arm support still needs some work. |
|
Any chance i can get a few pointers? I'm happy putting dev. time into it but since I'm not familiar with the code I'd appreciate some help. |
|
If you got through the protobuf issue, then the next step is to update the CROSSTOOL file to add an arm cpu + associated toolchain. |
Nah, still stucked there. |
|
Sorry to hear that. We're using checked-in protobuf binaries and runtimes, On Sat, Nov 28, 2015 at 4:27 PM, Víctor Mayoral Vilches <
|
|
@ulfjack thanks for your answer. I'm working on That said, I'm not doing any patching in the bootstrap script so that might be it. Could you provide a patch for arm? |
|
What kind of changes are you making to the source tree? Presumably you need On Mon, Nov 30, 2015 at 10:16 AM, Víctor Mayoral Vilches <
|
|
Note that we're using alpha-3 not beta-1; On Mon, Nov 30, 2015 at 10:31 AM, Ulf Adams ulfjack@google.com wrote:
|
|
To make sure we're on the same page, here's a walkthrough on my process:
|
|
Just learned that there's support only for 64 bit systems. Guess that explains it. |
|
What? Who or what only supports 64-bit systems? Java protobufs should work on any system that has a compliant JVM installed. |
|
Let me see if I can run this in an emulator... |
|
Ok, I have an ARM env in qemu now, using the instructions from here: |
|
That's great, thanks for setting that up @ulfjack.
Great to hear. |
|
Ok so seems like we have a bug in sandboxing on ARM |
|
You need to make sure that you use the bootstrapped binary, not the initial binary. Do we actually copy both of them to the same location? We should maybe more clearly indicate that the 'first' output/bazel binary isn't the right one. Maybe add a .preliminary suffix or something like that? |
|
We overwrite the initial one that is just here to bootstrap maybe we should On Tue, Dec 1, 2015, 7:27 PM Ulf Adams notifications@github.com wrote:
|
|
Trying to use what was generated before to get TensorFlow compiled with bazel but the process fails over and over: There seems to be small improvements every time I launch it.
This seems related. @ulfjack, I'm using the |
|
The |
|
Yeah, this error message means that sandboxing doesn't work: And the flags are disabling sandboxing: Can you make progress now? We should probably have a separate bug that sandboxing doesn't work on linux arm32. |
|
@ulfjack, I repeated the process several times and got the same result. Any chance you can advice on which flags/modifications can be made to make it compile? |
This seems to be the cause. Trying different options, |
|
Cool! How much memory do you have on the device? You probably want to reduce the number of jobs (--jobs=4 or maybe even fewer), and you can explicitly set how much memory should be available to the Java VM used by Bazel. |
|
I'm in a pi2:
This would be great. Could you point out how and where I can set the available memory to the Java virtual machine? This is what I'm getting so far: |
|
|
Oh, darn. That's the Javac invocation from the bootstrap, not bazel. The bootstrap compile bundles everything into a single javac run, and that might be pretty big. You should take a look at the files in scripts/bootstrap/, specifically compile.sh:212. |
You mean that |
|
No, it's most definitely RAM. I'm just saying that small tweaks may not be On Thu, Dec 3, 2015 at 4:16 PM, Víctor Mayoral Vilches <
|
|
Here's a summary of my failures so far (target platform: Raspberry Pi 2): Bazel/TensorFlow native compilation in the Raspberry Pi 2Thanks @danbri but still not enough resources. Tried with different swap sizes. Bazel cross-compiled, copied and used to build TensorFlow@ulfjack, tried cross-compilation also. Then Which seemed similar to what was described above. Tried to solve it as I did previously but failed. TensorFlow
|
|
It's not enough to copy over the bazel binary, you also need to copy over On Fri, Dec 4, 2015 at 4:24 PM, Víctor Mayoral Vilches <
|
|
@ulfjack thanks for the quick answer. Copied the whole |
|
@vmayoral perhaps you could post the error messages? |
|
Since this latest step is more a TensorFlow issue than a bazel one, i've opened a new issue at tensorflow/tensorflow#445. I'd say this issue has been addressed so we can close it. |
|
Hi @vmayoral, how about use Qemu + Docker to bring up the development environment ? I use it, but still have out of memory fail event. |
|
@philipz How did you get Java to work with QEMU? I tried running a build with a RPI image in QEMU, but Java consistently crashed on me. |
|
@ulfjack , Java and Golang both work, check out this great article. |
|
From a quick look, I did all that, but Java didn't work. Maybe I did something wrong somewhere? |
|
Distributor ID: Raspbian Bazel build release version: https://github.com/bazelbuild/bazel/releases/tag/0.5.2 I tried recently, getting error: ERROR: Could not build Bazel |
|
download the binary for armv7l(pi 3) with this link below |
Trying to compile bazel on a Raspberry Pi 2. To do so, i replaced
third_party/protobuf/protoc-linux-arm32.exewith the correspondingprotoccompiled from its sources (to get the 3.0.0 version). With this in mind i launchcompile.sh:https://gist.github.com/vmayoral/d2447888c5f5051a06db
I've reproduced it in a bootstrap image with
qemu-debootstrapin my dev. machine obtaining exactly the same result.Can anyone advice what I'm missing?
The text was updated successfully, but these errors were encountered: