pyinstaller-based binary for ARM (raspberry pi) #1018

nelg · 2016-05-05T21:17:38Z

Hi,

The pyinstaller-based binaries for borg are very useful. However, it would be nice if the was one available for the raspberry pi. (armv6l)

Glen

ThomasWaldmann · 2016-05-07T17:43:04Z

Not sure if creating a single-file binary is possible with pyinstaller, someone with a raspberry pi 1 would have to check. The development section of the docs describes how this is usually done, the Vagrantfile has some practically tested code.

Maybe these binaries could be also useful on other ARM based hardware, where installing borg otherwise is difficult to impossible (like some NAS devices).

ThomasWaldmann · 2016-05-07T17:46:35Z

BTW, a note about running on low-memory systems: borg needs quite some memory for the dedup hashtable and also for the files cache, so be careful. There's a formula in the docs to estimate RAM needs.

nelg · 2016-05-09T09:18:24Z

As a side note, I've run borg backup on arch on the raspberry pi, without major issues, backing up a fairly small set of files (< 2GB).

ThomasWaldmann · 2016-05-09T17:59:58Z

I removed all comments from this ticket dealing with the raspian package - dist packages are what #105 deals with.

bauerj · 2016-07-03T16:45:56Z

Not sure if creating a single-file binary is possible with pyinstaller, someone with a raspberry pi 1 would have to check.

I was able to successfully build a pyinstaller based armhf-binary of the 1.0.3 release 🎉 . You can download it here: borg-armhf (9.0MB) ⬇️

I did a quick test (only the basic commands) that showed the binary works on a raspberry pi without Python 3.

The linked libraries seem to be the ARM versions of what the official binaries are linked against:

bauerj@zero:~ $ ldd borg-armhf
        /usr/lib/arm-linux-gnueabihf/libarmmem.so (0xb6f40000)
        libdl.so.2 => /lib/arm-linux-gnueabihf/libdl.so.2 (0xb6f1b000)
        libz.so.1 => /lib/arm-linux-gnueabihf/libz.so.1 (0xb6ef4000)
        libc.so.6 => /lib/arm-linux-gnueabihf/libc.so.6 (0xb6db3000)
        /lib/ld-linux-armhf.so.3 (0x7f578000)

I built this binary on my Raspberry Pi so I was unable to use vagrant to build the binary (seems like using QEMU with vagrant wouldn't be a good idea 😯 ). The process was pretty straight-forward, although quite slow due to the limited amount of processing power and the need to compile most of the dependencies (including Python itself).

I guess the goal is to have official ARM binaries on the release page so the following issues need to be addressed:

It seems like you (the borg developers) don't have an ARM device which means that you can't generate the binary yourself. (Is that correct?)
The build process I used is very manual (e.g. copy-and-pasting the correct lines from the Vagrantfile).
Currently, unit tests are not run against ARM hardware (at least not automatically) so a failing ARM build would only be noticed after the release.

Any ideas?

enkore · 2016-07-03T16:54:05Z

While Vagrant + QEMU is probably not viable, Linux-on-QEMU should be. We used it before for powerpc (although that's very slow, but we got dedicated hardware now). Some more exotic tests (ie. not on Travis) tests are run here, if the speed of QEMU is feasible I might add it there.

One can also run a subset of the tests against a compiled Borg binary. It must be named borg.exe and be on the PATH for this. This is what the Vagrantfile also does.

I'm not sure if precompiled ARM binaries make a lot of sense, there are many ARM flavours. Since Borg is in Debian and most people seem to use that on RPis those should have packages available?

PlasmaPower · 2016-07-03T16:56:11Z

@enkore I'm assuming you're passing -enable-kvm to it? That will speed it up a lot.

bauerj · 2016-07-03T16:56:48Z

Sounds good, I think builds could also be produced on scaleway's ARM servers if QEMU turns out to be too slow.

bauerj · 2016-07-03T16:58:53Z

So the only issue that's left is finding a way to use the logic from the Vagrantfile without using Vagrant. As the syntax seems to be Ruby, maybe a Ruby script could be written?

PlasmaPower · 2016-07-03T17:00:09Z

Why exactly can't we use Vagrant? vagrant-libvirt is a thing (libvirt is a qemu manager).

bauerj · 2016-07-03T17:02:05Z

That project seems not to be stable yet and I couldn't find any ARM image for it.

bauerj · 2016-07-09T18:56:05Z

I'm not sure if precompiled ARM binaries make a lot of sense, there are many ARM flavours. Since Borg is in Debian and most people seem to use that on RPis those should have packages available?

Of course, with a Raspberry Pi you can just install python3, pip and then borg (it seems not to be included in Raspbian Jessie). My goal is to ultimately use borg on a NAS that's running a custom (Linux based) OS built by the hardware producer. There is no easy way to install python3 on that device and I don't want to break things by trying it.

I agree that that is probably an edge-case and most people who want to backup to such a device could just resort to rsync or even sftp. But you will probably agree that borg will outperform possible alternatives 😉

As to the different ARM flavours, most NAS seem to use armel, Raspberry Pi uses armhf. I guess those would make the most sense to have binaries for.

alexmbird · 2016-07-10T18:57:08Z

@bauerj before discovering this I spent an hour trying to roll my own pyinstaller binary, but couldn't get the resulting bin to work correctly. Could you post your version of pyinstaller and the command line you used?

bauerj · 2016-07-11T15:25:49Z

Sure. I'm using PyInstaller 3.1.1, this is essentially what I executed:

  ARCH=$(uname -m)
  PYTHON="3.5.2"
  curl -L https://raw.githubusercontent.com/yyuu/pyenv-installer/master/bin/pyenv-installer | bash
  apt-get install -y libssl-dev libacl1-dev liblz4-dev libfuse-dev fuse pkg-config
  apt-get install -y fakeroot build-essential git
  apt-get install -y zlib1g-dev libbz2-dev libncurses5-dev libreadline-dev liblzma-dev libsqlite3-dev
  env PYTHON_CONFIGURE_OPTS="--enable-shared" pyenv install -v $PYTHON
  pyenv shell $PYTHON
  cd /opt
  git clone https://github.com/borgbackup/borg/
  cd borg
  git checkout 1.0.3
  pip install -r requirements.d/development.txt
  cd /opt
  git clone https://github.com/pyinstaller/pyinstaller.git
  cd pyinstaller
  git checkout v3.1.1
  cd bootloader
  python ./waf all --no-lsb
  cd ..
  pip install -e .
  cd /opt/borg
  pip install -e .
  LD_LIBRARY_PATH=~/.pyenv/versions/$PYTHON/lib/ pyinstaller -F -n borg-$ARCH --clean borg/__main__.py

I hope that helps.

bmarwell · 2016-08-04T21:25:30Z

I'd like to test on armv7, banana pi. I installed open media vault, which only has python 3.2 support.

Can I cross compile from another computer, or do you have such a binary yet? Thanks.

bauerj · 2016-08-04T21:29:19Z

You can test this binary here: https://bauerj.eu/borg-armhf

Please report whether it worked or not.

bmarwell · 2016-08-05T05:12:52Z

Sadly, it did not.

root@openmediavault:~/Download# ./borg-armhf 
Error loading Python lib '/tmp/_MEIIgQFen/libpython3.5m.so.1.0': /lib/arm-linux-gnueabihf/libc.so.6: version `GLIBC_2.17' not found (required by /tmp/_MEIIgQFen/libpython3.5m.so.1.0)

ldd:

root@openmediavault:~/Download# ldd borg-armhf
        libdl.so.2 => /lib/arm-linux-gnueabihf/libdl.so.2 (0xb6f42000)
        libz.so.1 => /lib/arm-linux-gnueabihf/libz.so.1 (0xb6f28000)
        libc.so.6 => /lib/arm-linux-gnueabihf/libc.so.6 (0xb6e41000)
        /lib/ld-linux-armhf.so.3 (0xb6f55000)
        libgcc_s.so.1 => /lib/arm-linux-gnueabihf/libgcc_s.so.1 (0xb6e1d000)

Thanks for your efforts so far!

enkore · 2016-08-05T10:59:29Z

Try pyenv. I use it on the odroid C2 running the ARM64 tests. It compiles on the machine it's running, so it'll take a while.

bmarwell · 2016-08-05T11:12:44Z

Try pyenv. I use it on the odroid C2 running the ARM64 tests. It compiles on the machine it's running, so it'll take a while.

I'm not sure this is what I want. It takes a lot of time, upgrading is much more complicated. I'll see into it anyway.

bauerj · 2016-08-05T11:14:05Z

Well, as long as there are no official ARM binary builds it would be even harder to update the binary 😉

enkore · 2016-08-05T11:23:06Z

I'm not sure this is what I want. It takes a lot of time, upgrading is much more complicated. I'll see into it anyway.

I see what you mean. Especially on such weakly devices one would really rather have binary (Python, Borg) packages available. If that's not the case, though, then there are not that many options. I guess it's possible to cross-compile python, but it probably takes longer to setup the cross-compiling environment than just building it on the ARM.

iirc Python 3.x took about an hour or so on the C2 to build.

bmarwell · 2016-08-06T13:15:54Z

Pyenv and pip install took a while. Anyway, can I somehow provide a standalone binary to others?

bauerj · 2016-08-06T13:18:07Z

Sure!

Here are the commands I used to create a binary on the Raspberry Pi (of course you can skip the pyenv part). It would have worked on your device too if it wasn't for the old glibc version your OS uses.

bmarwell · 2016-08-08T12:28:33Z

@bauerj You were missing a pip install -e . in pyinstaller, otherwise it'd say "pyinstaller: not found".

Other than that, Banana Pi is armv7l. Please try this binary found in this folder. Please give me some information if this works for others: https://goo.gl/rEI49A.

kzar · 2016-10-07T21:25:37Z

@bmhm Your linked 1.0.6 armv7l build is the only one I've found that will run on my Synology DS416, thanks for that. Any chance you could create a new one for borg 1.0.7, that would mean I could avoid positional argument warnings.

bauerj · 2016-10-07T21:28:01Z

@kzar It would be nice if you could test the 1.0.7 binary here: https://borg.bauerj.eu/ (armv6 should hopefully work for you)

➡️ ➡️ ➡️ There will also be future versions avaiable here: 📂 https://borg.bauerj.eu/ ⬅️ ⬅️ ⬅️

kzar · 2016-10-07T21:38:43Z

@bauerj Wow, that's service! Thanks it seems to be running fine, my first borg backup is now underway. I'll let you know if it completes (will likely be some hours unfortunately).

kzar · 2016-10-07T22:10:13Z

@bauerj Well it keeps stalling when I attempt to do my first backup. It starts off happily and at some point silently stops transferring. I then have to C-c the process. It's likely nothing to do with your build, I'm planning to investigate a little more tomorrow to see if I can get it working.

kzar · 2016-10-10T09:25:20Z

@baurej My problem looks unrelated to your Borg build, rather issue #636. (I could reproduce the problem with just SCP too, it reliably drops after about 1gig has transferred.) I've not been able to do a big test with your build therefore, but it did at least run and work initially. Thanks again for the help, Dave.

enkore · 2016-10-10T09:40:16Z

If it happens with scp as well, then it's likely an issue outside of Borg.

Wild speculation:

1 GB could be the kernel running out of cache memory, so it starts writing to disk which is often slow with ARM boards
Perhaps a power dissipation problem? My odroid c2 board regularly locks up under load if no additional cooling (fan) is provided
More recent versions of SSH support chacha20-poly1305 which tends to be much faster than anything else on ARM (less CPU load, less heat). You can see what your connection is using by ssh -v ..., the lines kex: server->client cipher: and kex: client->server cipher: tell this.

kzar · 2016-10-19T14:41:53Z

OK so I found that if I connect via my VPN the connection doesn't drop, don't ask me why. @bauerj Your build seems to be working nicely so far.

bauerj · 2016-10-19T14:43:00Z

Great! 🎉 Thanks for testing.

ThomasWaldmann · 2017-01-22T01:37:13Z

@bauerj https://github.com/borgbackup/borg/pull/2076/files this change will be in 1.0.10*.

As you are building for ARM, you likely will compile the pyinstaller bootloader on your own (thus the inclusion of the fixed x64/x86 bootloaders makes no difference for you). But please make sure that the bootloader source you use has the same fixes.

It might be also worthwile doing an additional binary made with pyinstaller 3.2.1.

bauerj · 2017-01-22T08:11:49Z

Thanks for the heads-up!

I don't know how to interpret the last sentence though. Should I do another 1.0.9 binary with pyinstaller 3.2.1 or should I do two releases (one with your fork and one with pyinstaller 3.2.1) for 1.0.10?

ThomasWaldmann · 2017-01-22T14:03:08Z

I meant the upcoming 1.0.10 release (likely in a few days).
I will do the official release (x86/x64) binaries using the current Vagrantfile.
Additionally, I'll do some experimental x64 binary using pyi 3.2.1.

milkey-mouse · 2017-07-19T21:38:14Z

FWIW you could always cross-compile for ARM using something like this.

kzar · 2017-11-08T10:39:02Z

Any chance of a 1.1.X build @bauerj?

bauerj · 2017-11-08T10:43:28Z

There you go: https://borg.bauerj.eu/

kzar · 2017-11-08T10:54:48Z

Awesome, thanks. Seems to work so far, doing a backup now.

noplanman · 2018-09-28T09:29:12Z

Just to let everybody here know, that I've managed to set up a docker solution for building binaries for all types of architectures.

Take a look here: https://git.feneas.org/noplanman/borg-binary-builder

I'm also looking into a solution for providing signed binaries directly, without requiring to do the whole docker operation for every new version.

ThomasWaldmann · 2018-12-15T23:09:46Z

as the arm binaries are linked from the docs now, the borg-binary-builder from community repo, i guess there is nothing left to do for this ticket.

ThomasWaldmann changed the title ~~pre-compiled for arm (raspberry pi)~~ pyinstaller-based binary for ARM (raspberry pi) May 7, 2016

ThomasWaldmann added the help wanted label May 9, 2016

enkore mentioned this issue Jul 11, 2016

package Borg for OpenWRT #1311

Closed

friesenkiwi mentioned this issue Jul 1, 2017

Package for systems with uClibc bauerj/borg-binaries#3

Closed

infectormp mentioned this issue Nov 12, 2018

Standalone binaries seem to be x32/64 only #4165

Closed

ThomasWaldmann closed this as completed Dec 15, 2018

pyinstaller-based binary for ARM (raspberry pi) #1018

pyinstaller-based binary for ARM (raspberry pi) #1018

Comments

nelg commented May 5, 2016 • edited by ThomasWaldmann Loading

ThomasWaldmann commented May 7, 2016 • edited Loading

ThomasWaldmann commented May 7, 2016 • edited Loading

nelg commented May 9, 2016 • edited by ThomasWaldmann Loading

ThomasWaldmann commented May 9, 2016 • edited Loading

bauerj commented Jul 3, 2016 • edited Loading

enkore commented Jul 3, 2016 • edited Loading

PlasmaPower commented Jul 3, 2016

bauerj commented Jul 3, 2016

bauerj commented Jul 3, 2016

PlasmaPower commented Jul 3, 2016

bauerj commented Jul 3, 2016

bauerj commented Jul 9, 2016 • edited Loading

alexmbird commented Jul 10, 2016

bauerj commented Jul 11, 2016 • edited Loading

bmarwell commented Aug 4, 2016

bauerj commented Aug 4, 2016

bmarwell commented Aug 5, 2016

enkore commented Aug 5, 2016 • edited Loading

bmarwell commented Aug 5, 2016

bauerj commented Aug 5, 2016

enkore commented Aug 5, 2016 • edited Loading

bmarwell commented Aug 6, 2016

bauerj commented Aug 6, 2016 • edited Loading

bmarwell commented Aug 8, 2016 • edited Loading

kzar commented Oct 7, 2016

bauerj commented Oct 7, 2016 • edited Loading

kzar commented Oct 7, 2016

kzar commented Oct 7, 2016

kzar commented Oct 10, 2016

enkore commented Oct 10, 2016

kzar commented Oct 19, 2016

bauerj commented Oct 19, 2016

ThomasWaldmann commented Jan 22, 2017

bauerj commented Jan 22, 2017

ThomasWaldmann commented Jan 22, 2017

milkey-mouse commented Jul 19, 2017

kzar commented Nov 8, 2017

bauerj commented Nov 8, 2017

kzar commented Nov 8, 2017

noplanman commented Sep 28, 2018

ThomasWaldmann commented Dec 15, 2018

nelg commented May 5, 2016 •

edited by ThomasWaldmann

Loading

ThomasWaldmann commented May 7, 2016 •

edited

Loading

ThomasWaldmann commented May 7, 2016 •

edited

Loading

nelg commented May 9, 2016 •

edited by ThomasWaldmann

Loading

ThomasWaldmann commented May 9, 2016 •

edited

Loading

bauerj commented Jul 3, 2016 •

edited

Loading

enkore commented Jul 3, 2016 •

edited

Loading

bauerj commented Jul 9, 2016 •

edited

Loading

bauerj commented Jul 11, 2016 •

edited

Loading

enkore commented Aug 5, 2016 •

edited

Loading

enkore commented Aug 5, 2016 •

edited

Loading

bauerj commented Aug 6, 2016 •

edited

Loading

bmarwell commented Aug 8, 2016 •

edited

Loading

bauerj commented Oct 7, 2016 •

edited

Loading