Any chance to get aarch64 for 1.13.1 too? #10

marianopeck · 2019-07-11T02:23:59Z

[Required] RaspberryPi3 / Jetson Nano

[Required] armv7l and aarch64

[Required] Raspbian, Ubuntu Tegra

[Required] Overview of problems and questions:

I have a current binding against 1.13.1. I see there is only support for armv7l. But I would love to have to aarch64 too. I see 1.14.1 does have support for both, but it would imply upgrading the binding I am using.

Any chance for this to happen?

PINTO0309 · 2019-07-11T03:41:52Z

As a hobby volunteer I work on a weekend, I do not have enough working time to meet the needs of everyone. So, for now, I'm lowering the build priority of older versions.

Build priority

1. Tensorflow v1.14.0 armhf - Raspbian (Debian) Buster ... Currently in progress
2. Tensorflow v2.0.0-beta1 armhf - Raspbian (Debian) Buster
3. Tensorflow v2.0.0-beta1 aarch64 - Debian Buster
4. etc...

If you are in a hurry, consider building for yourself.
https://github.com/PINTO0309/Tensorflow-bin#build-parameter

or

Jetson Nano
https://dl.photoprism.org/tensorflow/

marianopeck · 2019-07-11T14:25:56Z

Hi @PINTO0309

Thanks for your response. You work is very much appreciated :)

I think I misunderstood the resulting pre binaries...I was looking for C binaries (.so) for Raspbian and aarch64. You can see I asked here and here but I didn't get much answers except "compile myself", which I was trying to avoid so far.

Are you aware of someone shipping pre-build C libraries (.so) for ARM and ARM64?

Thanks!

PINTO0309 · 2019-07-11T15:10:47Z

@marianopeck
Unfortunately I do not know. If I had time to search for .so, I think it would be faster to build it myself.

marianopeck · 2019-07-15T17:22:51Z

Hi @PINTO0309

Thanks for your reply. I understand your point. And in fact, I have already started to try to compile myself. In the meanwhile, while I undertand that adding new versions/platform combinations is too much effort, did you consider to add support to not only build the Python wheel but also a shared library? As far as I understand, everything is exactly the same except the line:

//tensorflow/tools/pip_package:build_pip_package

which should be:

//tensorflow/tools/lib_package:libtensorflow

Thoughts?

PINTO0309 · 2019-07-17T03:58:01Z

@marianopeck
I know that you can create shared libraries if you build in the way you suggested.
https://stackoverflow.com/questions/49788062/how-to-build-the-c-library-for-tensorflow

At the moment, I'm building only the Wheel file to make working with Python users much easier.
I'm interested, so when I have time, I will try building a shared library.

marianopeck · 2019-07-17T11:46:34Z

HI @PINTO0309

Thanks for considering building the shared libraries. It's not minimize the great work you are doing, but just in case you didn't know, since a couple of releases there are OFFICIAL (from Google) Wheel even for the Raspberry. See the "Raspberry Pi" tab.

However, they still do NOT offer shared libraries for the Pi. So, IMHO it would be more valuable to have shared libraries that Wheels, as Wheels are available officially.

Best regards,

PINTO0309 · 2019-07-17T13:36:15Z

Hi, @marianopeck
The difference between my Wheel file and the official Wheel is the following two points.

Official Wheel is disabled by default on Tensorflow Lite. However, my wheel is enabled by default.
Tuned to 2.5 times the performance of the official Tensorflow Lite.

It is 1. that is considered a problem by engineers around the world.

marianopeck · 2019-07-17T14:23:05Z

Wow. Thanks @PINTO0309 for the details!! I didn't know that.
BTW, how did you increase 2.5 times the performance? Just curious.
Thanks!

PINTO0309 · 2019-07-17T14:38:08Z

@marianopeck
The implementation method is described below.
https://github.com/PINTO0309/Tensorflow-bin#build-parameter

The experimental results are described below.
https://github.com/PINTO0309/Tensorflow-bin#operation-check

Attention point is the following.

I'm only tuning the Python API.
Performance is maximized only in the convolutional layer.
It performs CPU inference using up to 4 threads of multi-threads, but performance will deteriorate if executed with 5 or more threads.
Performance is worse with Jetson. Simply infer using the GPU with TensorRT.

marianopeck · 2019-07-17T15:37:47Z

Hi @PINTO0309
Tanks for your detailed answer.
I finally took the time to write down everything I found during my attempt to get TensorFlow C library compiled for Raspberry Pi. In that tutorial, I point back to this issue in case you eventually provide shared libraries.
Thank you

PINTO0309 · 2019-07-17T21:03:21Z

@marianopeck

You can find Bazel's pre-built binaries below.
https://github.com/PINTO0309/Bazel_bin

I saw an article about your great effort. I have been helped by freedomtan as well as you. By the way, I built an RaspberryPi emulation environment with QEMU 4.0.0 to solve the problem of SDCard corruption due to insufficient RAM and SWAP.

marianopeck · 2019-07-18T01:45:35Z

Wow....that's pretty cool!!!! It would have save me quite some time. I will update my post with this info. Do you know if it is possible to know in advance which bazel version will a given version of TensorFlow need?
Thanks!

PINTO0309 · 2019-07-18T03:23:28Z

@marianopeck

For example, for Tensorflow v1.14.0:

tensorflow/tensorflow/tools/ci_build/install/install_bazel.sh

# Select bazel version.
BAZEL_VERSION="0.24.1"

set +e
local_bazel_ver=$(bazel version 2>&1 | grep -i label | awk '{print $3}')

if [[ "$local_bazel_ver" == "$BAZEL_VERSION" ]]; then
  exit 0
fi

For example, for Tensorflow v1.13.2:

# Select bazel version.
BAZEL_VERSION="0.20.0"

set +e
local_bazel_ver=$(bazel version 2>&1 | grep -i label | awk '{print $3}')

if [[ "$local_bazel_ver" == "$BAZEL_VERSION" ]]; then
  exit 0
fi

marianopeck · 2019-07-18T03:26:45Z

You rock! I will be adding this info in by post soon...thanks!

PINTO0309 · 2019-07-20T06:15:46Z

@marianopeck
I tried to create a C binary of v1.14.0.
https://github.com/PINTO0309/Tensorflow-bin/tree/master/C-library/1.14.0-armv7l

Usage

$ wget https://github.com/PINTO0309/Tensorflow-bin/raw/master/C-library/1.14.0-armv7l/libtensorflow.tar.gz
$ tar -C /usr/local -xzf libtensorflow.tar.gz
$ rm libtensorflow.tar.gz
$ sudo ldconfig

marianopeck · 2019-07-21T17:28:59Z

Hi @PINTO0309

That worked perfectly!! Does that means you will start shipping them too? :)

marianopeck · 2019-07-21T17:47:26Z

@PINTO0309 BTW, I also have 64 bits OS running on the Pi3 (like Armbian and Ubuntu Server)...so...if you give me a aarch64, I can also test them there! Would love to get aarch64 too :)

PINTO0309 · 2019-07-21T22:21:14Z

@marianopeck
I’m very busy on weekdays so I don’t have enough work time. Please wait until the weekend. I will proceed with the work little by little.

marianopeck · 2019-07-22T12:40:17Z

Hi @PINTO0309
I have no rush from my end. I was able to compile it from scratch myself (on ARM 32 bits) and I guess I will give it a try soon on ARM 64 too. Why questions (both, if you would start providing this as well as aarch64) is more for a long term perspective.
Thanks a lot for your work!

PINTO0309 · 2019-07-22T15:10:45Z

@marianopeck
I will start building aarch64 from now. I will continue this volunteer until I get bored with the build process.

PINTO0309 · 2019-07-23T12:43:28Z

@marianopeck
I have released Tensorflow v1.14.0 C-Library aarch64.
https://github.com/PINTO0309/Tensorflow-bin/tree/master/C-library/1.14.0-aarch64

marianopeck · 2019-07-23T16:24:27Z

Hi @PINTO0309
Thanks a lot for your work. I just gave it a try but the ldd is failing to find the correct version of glibc:

ubuntu@ubuntu:~/Instantiations/tensorflow/PINT/lib$ ldd libtensorflow.so.1.14.0 ./libtensorflow.so.1.14.0: /lib/aarch64-linux-gnu/libc.so.6: version `GLIBC_2.28' not found (required by ./libtensorflow.so.1.14.0)
	linux-vdso.so.1 (0x0000ffff9a5d2000)
	libtensorflow_framework.so.1 => /home/ubuntu/Instantiations/tensorflow/PINT/lib/./libtensorflow_framework.so.1 (0x0000ffff9468c000)
	libpthread.so.0 => /lib/aarch64-linux-gnu/libpthread.so.0 (0x0000ffff9464e000)
	libdl.so.2 => /lib/aarch64-linux-gnu/libdl.so.2 (0x0000ffff94639000)
	libm.so.6 => /lib/aarch64-linux-gnu/libm.so.6 (0x0000ffff9457f000)
	libstdc++.so.6 => /usr/lib/aarch64-linux-gnu/libstdc++.so.6 (0x0000ffff943ec000)
	libgcc_s.so.1 => /lib/aarch64-linux-gnu/libgcc_s.so.1 (0x0000ffff943c8000)
	libc.so.6 => /lib/aarch64-linux-gnu/libc.so.6 (0x0000ffff9426f000)
	/lib/ld-linux-aarch64.so.1 (0x0000ffff9a5a7000)

I am under Ubuntu 18.04:

uname -a
Linux ubuntu 4.15.0-1037-raspi2 #39-Ubuntu SMP PREEMPT Wed May 15 20:15:44 UTC 2019 aarch64 aarch64 aarch64 GNU/Linux

And:

 cat /etc/os-release 
NAME="Ubuntu"
VERSION="18.04.2 LTS (Bionic Beaver)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 18.04.2 LTS"
VERSION_ID="18.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=bionic
UBUNTU_CODENAME=bionic

I am using glibc 2.27.

 ldd --version
ldd (Ubuntu GLIBC 2.27-3ubuntu1) 2.27
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Written by Roland McGrath and Ulrich Drepper.

I already did a:

  sudo apt-get update
  sudo apt-get upgrade

I have searched SO a lot but I cannot find an easy way to workaround this. Any idea?

marianopeck · 2019-07-23T16:37:40Z

There is no way to just link to libc or libc.so.6 or whatever is found in my machine? or TF would only work with concrete glibc versions?

PINTO0309 · 2019-07-23T16:39:42Z

@marianopeck
Ah, sorry... I have built on Debian Buster aarch64... The Debian Buster glibc seems to be 2.28.
https://packages.debian.org/search?keywords=glibc&searchon=names&suite=buster&section=all

In order to make a binary that can be used smartly in your environment, it may be early to rebuild with Ubuntu 18.04. I need a lot of time to prepare the environment.

marianopeck · 2019-07-23T16:48:30Z

Hi @PINTO0309
Yeah, I imagined that was the reason. But haven't you to build on each different glibc is a pain. As well as for shipping/distributing.. I mean, would you need one version per glibc version? There is no way to link agains some more generic libc.so.6?

For example, I just tried on my other device, a Rock64 with Armbian 64 bits. And it has glibc 2.24. So it won't work there either. I am checking with the official linux distro and it seems more flexible:

      ldd libtensorflow.so.1.14.0 | grep libc                                                              
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fc4eb652000)

PINTO0309 · 2019-07-23T17:02:46Z

I think that building glibc on my own seems to be a quick solution.

marianopeck · 2019-08-05T14:28:24Z

Attention point is the following.

I'm only tuning the Python API.

Performance is maximized only in the convolutional layer.

It performs CPU inference using up to 4 threads of multi-threads, but performance will deteriorate if executed with 5 or more threads.

Performance is worse with Jetson. Simply infer using the GPU with TensorRT.

Hi @PINTO0309
I am thinking on what you wrote above...because you wrote that BEFORE shipping shared library. So...let me ask...is that performance tuning you are doing still only for Python? If so, is there a chance you apply the same for the C shared libraries?

Thanks!

snowzach · 2019-08-17T20:48:23Z

Since you guys are the professionals here... I have questions.
I am trying to build these 3 libraries for armhf and aarch64

libtensorflowlite.so
libtensorflowlite_c.so
libtensorflow.so

So I can support this project: https://github.com/snowzach/doods
I have a Pi 3b, Pi4 4G and an Odroid C2 I want to create docker containers for.
If there's easyish instructions to do this somewhere, I would be greatly appreciative.

marianopeck · 2019-08-17T21:33:37Z

HI @snowzach

I wrote the best I could here and here. I couldn't write yet, but I also succeeded on Rock64 with something like this:

sudo apt-get install gcc g++ swig build-essential openjdk-11-jdk python zip unzip
sudo apt install python-pip python-setuptools
pip install future


env BAZEL_JAVAC_OPTS="-J-Xms384m -J-Xmx4096m" \
JAVA_TOOL_OPTS="-Xmx4096m" \
EXTRA_BAZEL_ARGS="--host_javabase=@local_jdk//:jdk" \
bash ./compile.sh


bazel --host_jvm_args=-Xmx4096m --host_jvm_args=-Xms384m build \
--config opt --verbose_failures --local_resources 4096,3.0,1.0 \
//tensorflow/tools/lib_package:libtensorflow

With tensorflow 14.0 and bazel 24

snowzach · 2019-08-21T13:36:14Z

Thanks @marianopeck and @PINTO0309 between you two guy's resources, I have managed to get my container building for the pi. I am waiting for my SDCard so I can do an aarch64 build next for my odroid c2.

marianopeck · 2019-08-21T13:44:30Z

Excellent! Just curious, which aarch64 OS are you planning to use for the odroid?

…

On Wed, Aug 21, 2019, 10:36 Zach ***@***.***> wrote: Thanks @marianopeck <https://github.com/marianopeck> and @PINTO0309 <https://github.com/PINTO0309> between you two guy's resources, I have managed to get my container building for the pi. I am waiting for my SDCard so I can do an aarch64 build next for my odroid c2. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#10?email_source=notifications&email_token=AAH4FATIO3UV5B4GPLG54C3QFVAE7A5CNFSM4IAMJRF2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD4ZV3UQ#issuecomment-523460050>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAH4FAVFG5XORB4BA3S7CALQFVAE7ANCNFSM4IAMJRFQ> .

snowzach · 2019-08-21T13:48:21Z

@marianopeck armbian buster seems to be the easiest thus far.
https://www.armbian.com/odroid-c2/
I have based all of my builds off of buster. It's kinda large but I'm packing a lot of libraries in there.

marianopeck · 2019-08-21T14:20:03Z

I was gonna suggest that too :) I used Armbian Buster aarch64 in Rock64 and it worked great :)

…

On Wed, Aug 21, 2019 at 10:48 AM Zach ***@***.***> wrote: @marianopeck <https://github.com/marianopeck> armbian buster seems to be the easiest thus far. https://www.armbian.com/odroid-c2/ I have based all of my builds off of buster. It's kinda large but I'm packing a lot of libraries in there. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#10?email_source=notifications&email_token=AAH4FAXFNBFSSNJQXQDLIILQFVBSNA5CNFSM4IAMJRF2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD4ZXDUI#issuecomment-523465169>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAH4FARQAW6OOAPTMQ3YELDQFVBSNANCNFSM4IAMJRFQ> .

-- Mariano Martinez Peck Email: marianopeck@gmail.com Twitter: @MartinezPeck LinkedIn: www.linkedin.com/in/mariano-martinez-peck <https://www.linkedin.com/in/mariano-mart%C3%ADnez-peck/> Blog: https://marianopeck.wordpress.com/

PINTO0309 · 2020-07-13T15:51:40Z

As there has been no progress, I'm closing it.

suggfudge · 2022-10-18T11:16:21Z

Hi, I just picked up some guides to start generating wheels for 32bit raspbian buster, I cant find where in this process do I start linking to .so files.

PINTO0309 mentioned this issue Aug 16, 2019

import error on jetson nano #12

Closed

PINTO0309 mentioned this issue Oct 2, 2019

'GLIBC_2.29' error not found for Python 3.7 + Tensorflow v2 #17

Closed

PINTO0309 closed this as completed Jul 13, 2020

Any chance to get aarch64 for 1.13.1 too? #10

Any chance to get aarch64 for 1.13.1 too? #10

Comments

marianopeck commented Jul 11, 2019

PINTO0309 commented Jul 11, 2019 • edited

marianopeck commented Jul 11, 2019

PINTO0309 commented Jul 11, 2019

marianopeck commented Jul 15, 2019

PINTO0309 commented Jul 17, 2019

marianopeck commented Jul 17, 2019

PINTO0309 commented Jul 17, 2019 • edited

marianopeck commented Jul 17, 2019

PINTO0309 commented Jul 17, 2019 • edited

marianopeck commented Jul 17, 2019

PINTO0309 commented Jul 17, 2019 • edited

marianopeck commented Jul 18, 2019

PINTO0309 commented Jul 18, 2019

marianopeck commented Jul 18, 2019

PINTO0309 commented Jul 20, 2019 • edited

marianopeck commented Jul 21, 2019

marianopeck commented Jul 21, 2019

PINTO0309 commented Jul 21, 2019

marianopeck commented Jul 22, 2019

PINTO0309 commented Jul 22, 2019 • edited

PINTO0309 commented Jul 23, 2019

marianopeck commented Jul 23, 2019

marianopeck commented Jul 23, 2019

PINTO0309 commented Jul 23, 2019

marianopeck commented Jul 23, 2019

PINTO0309 commented Jul 23, 2019

marianopeck commented Aug 5, 2019

snowzach commented Aug 17, 2019

marianopeck commented Aug 17, 2019

snowzach commented Aug 21, 2019

marianopeck commented Aug 21, 2019 via email

snowzach commented Aug 21, 2019

marianopeck commented Aug 21, 2019 via email

PINTO0309 commented Jul 13, 2020

suggfudge commented Oct 18, 2022

PINTO0309 commented Jul 11, 2019 •

edited

PINTO0309 commented Jul 17, 2019 •

edited

PINTO0309 commented Jul 17, 2019 •

edited

PINTO0309 commented Jul 17, 2019 •

edited

PINTO0309 commented Jul 20, 2019 •

edited

PINTO0309 commented Jul 22, 2019 •

edited