Synology installation

[gaetan-be]

Hello,

I tried installing borgbackup on a Synology NAS.
I got an error about OpenSSL headers

STORAGE> python3 -m pip install borgbackup
Collecting borgbackup
  Using cached borgbackup-1.0.0.tar.gz
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-build-elizlilb/borgbackup/setup.py", line 111, in <module>
        raise Exception('Unable to find OpenSSL >= 1.0 headers. (Looked here: {})'.format(', '.join(possible_openssl_prefixes)))
    Exception: Unable to find OpenSSL >= 1.0 headers. (Looked here: /usr, /usr/local, /usr/local/opt/openssl, /usr/local/ssl, /usr/local/openssl, /usr/local/borg, /opt/local)

Any clue?

[ThomasWaldmann]

Well, you need to have the openssl headers installed (they are needed for compiling software that uses openssl lib). And they need to be at the place it told, see "Looked here:".

Besides these, you will also need some other headers files, see the docs of borgbackup for details.

[ThomasWaldmann]

Instead of trying the source based install, you can also try the single-file binaries (see github releases page), they don't need compiling (but you'll need to have a x86/x64 cpu and a not-too-old glibc).

[ThomasWaldmann]

btw, if installation on the NAS is too painful, you can also use a network filesystem mounted on the client and just run borg on the client and backup into that mounted fs. running borg as client/server is more efficient, but this method can be used as a fallback in case c/s is not possible.

[level323]

Specifically in the case of Synology NAS'es there is also the debian-chroot package that can be installed on the NAS by adding the SynoCommunity package repository via the Disk Station Manager (DSM). This gives you a "full" debian installation that you can play in - avoiding the twin terrors of mucking around with the actual OS that is running your NAS and dealing with trying to get borg to install/run in that stripped-down host OS. I can confirm that the pre-built borg packages work for me out-of-the-box on my Synology NAS inside the debian-chroot. However, note that my Synology NAS uses an Intel processor. Lower-end Synology NAS'es use ARM CPU's and in this case you'll need to build/install borg manually. However, my understanding is that the SynoCommunity debian-chroot will install on non-Intel Synology NAS'es, which means that the build/install process inside that chroot will be relatively straightforward, as it is for any debian system.

[gaetan-be]

Thanks for the extensive answers, I will try out your solutions and give you feedback.

Borg backup on a NAS seems to be a natural fit to me. It would be great to have packages readily available for the 2-3 big names in the NAS market.

[snhrdt]

Personally, I use a Ubuntu docker image on my Synology NAS, with openssl, openssh, anacron, and borg installed. Works like a charm!

[dfloyd888]

As a newcomer, this is a useful topic, as having borgbackup on the NAS provides a layer of protection against ransomware (since the files are stored where a client can't access them.) There are different tiers of Synology NAS models. Some are x86 compatible, others use ARM CPUs, so what might work with a higher end NAS may be tough to do with one of the "j" models.

[ThomasWaldmann]

If there is some good / proven way to run borg on Synology NASes, I'ld appreciate if someone writes up something short for the FAQ section of our docs. Or puts up a howto somewhere on the web, where we could link to from the FAQ (if not so short).

As this is not a borg issue per se, I am closing this.

[fr34k8]

@ThomasWaldmann closed is right here. the 64bit latest binary works on 916+ synology.

ash-4.3# uname -a
Linux Syno916plus 3.10.77 #7393 SMP 2016 x86_64 GNU/Linux synology_braswell_916+

You could give us some ARM build here, to support Synology at it's best.
https://github.com/borgbackup/borg/releases offers no ARM build right?

Synology has plenty of "armv7l" => 3.2.40 devices

[smithjcm]

I use Borgmatic wrapper and Borgbackup binary running on an Ubuntu 1404 machine backing up to a Synology 1813+ (x86) NAS, which is running the same Borgbackup binary. Ubuntu machine backs up to the Synology NAS in minutes, really impressive backup solution.

[fr34k8]

@smithjcm how much data in there ?

[smithjcm]

Nightly it takes a couple of minutes to add a few hundred megabytes of changes to the 1.75TB backup

Duration: 2 minutes 3.59 seconds
Number of files: 382266
------------------------------------------------------------------------------
                       Original size      Compressed size    Deduplicated size
This archive:                2.17 TB              1.99 TB            212.66 MB
All archives:               16.64 TB             14.89 TB              1.75 TB

                       Unique chunks         Total chunks
Chunk index:                 1364582             10784618

A big initial backup took 18 hours

Duration: 18 hours 16 minutes 50.44 seconds
Number of files: 865561
------------------------------------------------------------------------------
                       Original size      Compressed size    Deduplicated size
This archive:                2.38 TB              2.13 TB              1.57 TB
All archives:                3.61 TB              2.97 TB              1.75 TB

                       Unique chunks         Total chunks
Chunk index:                 1359054              2497616
------------------------------------------------------------------------------

18 hours is a long time but I backup the same files to the same NAS with Duplicity and Duplicity takes a little over twice as long

[enkore]

25-30 MB/s could already be the limit what the NAS can shovel through an SSH channel, if it has no hardware acceleration. There are some faster crypto options there, I think recent SSH supports poly1305-chach20.

[markhor75]

Hi, @nprncbl can you said more about your method ? I think you mount (read only ?) the shares to backup or mayby all /volume1 as volume in the container ?
Thanks

[mmatiaschek]

I have a Synology Diskstation 212+ with current Diskstation Version 6.1.1 and i can't get neither sshfs nor borgbackup to run.

Linux ... 2.6.32.12 #15101 Wed Apr 26 14:42:28 CST 2017 armv5tel GNU/Linux synology_88f6282_212+

I guess i'll have to stick with the rsync with hardlinks solution for a while...

[level323]

@mmatiaschek There are a couple of issues you may like to consider in your case:

• Running borgbackup on a Diskstation 212+ could be problematic anyway, as it's specs are on the low end. I'm thinking specifically of the RAM on this unit, which is only 512MB. Running borg on this machine could run into out-of-memory issues, particularly if you're backing up multi-terabytes.

• The CPU on this system is ARM, so the only way to get borg running on it is to compile it yourself. Compiling directly on the 212+ host OS would not be recommended. Cross-compiling on another machine to ARMv5 is technically possible, but would be quite a challenge. Compiling the self-contained binary on via an emulator might work. Probably the easiest approach is via the debian-chroot package (see my earlier comment) assuming it's available for this particular model.

However, the most elegant solution IMO would be to run everything on the source machine (the machine that you're backing up), as mentioned by @ThomasWaldmann in an earlier comment. To summarise: mount the borg repo (that resides on the Synology) via a suitable mechanism (e.g. CIFS which the 212+ supports out-of-the-box). Then run borg on the source machine, specifying the path to the mounted repo. This should work well. No need to stick with rsync+hardlinks.

[ncoif]

Maybe late to the party, but here are some borg binary compiled for arm (no idea how he did it though, and haven't tried it):

https://borg.bauerj.eu/

[thunfischbrot]

Found the same after some searching. They do work well!

[noplanman]

🎉 Borg is now officially available for Synology from the SynoCommunity repo:
https://synocommunity.com/package/borgbackup

(Still needs a custom build/install of Python3 though, as it requires version 3.5.2, but the latest on the repo is 3.4.1.)

[ppawlak]

@noplanman How do you build or install a custom Python 3.5?

[noplanman]

@ppawlak It's very simple!

First, get your docker container set up:
https://github.com/SynoCommunity/spksrc#docker

Next, check which CPU architecture your DSM has:
https://github.com/SynoCommunity/spksrc/wiki/Architecture-per-Synology-model

Let's assume you have a DS416. Searching for that on the page, you'll find it under alpine, which is the CPU architecture.

Then, inside your docker container:

# List all available toolchains, showing only available ones for your architecture (e.g. "alpine")
$ ls toolchains | grep alpine

# Set the toolchain to be used, which is the DSM version you're running (any version that is available from the previous command):
# make dsm-{DSM_VERSION}
$ make dsm-6.1

# Change into the folder of the package you want to build (e.g. python3)
$ cd spk/python3

# Build for your architecture! (e.g. "alpine"):
# make arch-{ARCHITECTURE}
$ make arch-alpine

After the build has finished, you can find the package in the spksrc/packages folder, ready to be installed manually on your DS.

❗ Note: If you run into permission issues, check here:
SynoCommunity/spksrc#2733 (comment)

Then, after Python3 is successfully installed, you can install Borg from the "Community" list in the DSM Package Center.

Once that's done, have some well deserved cookies 😋 🍪

[ThomasWaldmann]

@noplanman do you know why they went for 3.5.2 instead using 3.4? borg should work with 3.4.

[noplanman]

@ThomasWaldmann That's a very good question, I don't know.

I'll try to lower the constraint to the SynoCommunity's Python 3 (v3.4.1-4) and build Borg and test it 👍
(Simply lowering the version in the text file of the package didn't work 😁 Worth a try though!)

[ThomasWaldmann]

Maybe someone should also push the python maintainer of SynoCommunity a bit. :)

Usually the early dot releases like .0 or .1 are worse than just taking the current one, which is 3.4.8.

The python devs are rather careful, so compatibility issues are not expected, but bug and security fixes of the most recent dot release are nice to have (and bugs in early dot releases are sometimes really annoying).

[noplanman]

Lowering the constraint did not work 😕
I'm not sure how to lower it "properly", so that Python 3.4 is used for the whole build process, as I assume that it still uses 3.5 internally, hmm...

But, I guess it shouldn't really be an issue once they update their official Python 3 package to 3.5, then this custom build and install will become unnecessary.

[wzyboy]

I followed the instructions of SynoCommunity above and successfully installed BorgBackup on Synology DiskStation.

A few words: one could create a new user with minimal privileges in DiskStation for backup purpose only. However, by default only users in admin group are allowed to login with SSH. The trick to overcome this is simple: change the login shell from /sbin/nologin to /bin/sh for the newly-created user. There is no chsh tool in DSM so you have to vim /etc/passwd and do it manually (with caution!).

For key-based SSH login, another trick is needed. If you use ssh-copy-id to copy your public key to the server, you still cannot log in because the weird umask settings in DSM makes everything of 777 permission. You need to chmod 700 ~; chmod 700 ~/.ssh; chmod 600 ~/.ssh/authorized_keys, or sshd would not let you in.

With these two tricks, a DSM is no different than a normal Linux box.

[wzyboy]

On more thing: borg is in $PATH for login shells. But you need to use --remote-path /usr/local/bin/borg for non-interactive access (e.g. borgmatic).

[noplanman]

@wzyboy No need to build Python3 and Borg yourself, as the official packages should be available. Best to take the officially signed ones!

What DiskStation do you have?

[wzyboy]

@noplanman Yes. I tried the offical single-file-binary and it runs well on the DiskStation. But I prefer the SynoCommunity package because it is supposed to be easier to update.

I do not know the exact model of the DiskStation as it's an office property (I never had a close look at it). But AFAIK it's an x86_64 model with 4 disks and 8 GiB RAM.

[wzyboy]

@noplanman Hmm, I read your comments above again. You said you need a custom build of Python. But today after I added the SynoCommunity repo and clicked "Install" on "Borg", the package manager installed "Python3" as a dependency automatically for me. It seems that they have already updated the "Python3" package to Python 3.5:

borg@DiskStation:~$ /volume1/@appstore/python3/bin/python3.5
Python 3.5.2 (default, Apr 25 2018, 20:32:51) 
[GCC 4.7.3 20130102 (prerelease)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> 

[noplanman]

@wzyboy Right, I must have not been clear enough in my comment.
I was trying to say that it's better to use the official package provided by SynoCommunity, instead of building it yourself.

But as you've explained, that's what you have already, so perfect! 👍

And yes, installing a package that depends on another one, will install the dependency automatically.

I need to look into the SSH tip you mentioned above, which I'm already very grateful for, as I might be running into that when I start playing with it 😉

[noplanman]

Just pointing to the latest PR regarding a BorgBackup update for Synology:
SynoCommunity/spksrc#3471 (for version 1.1.7)
SynoCommunity/spksrc#3418 (for version 1.1.6)

[scruel]

I followed the instructions of SynoCommunity above and successfully installed BorgBackup on Synology DiskStation.

A few words: one could create a new user with minimal privileges in DiskStation for backup purpose only. However, by default only users in admin group are allowed to login with SSH. The trick to overcome this is simple: change the login shell from /sbin/nologin to /bin/sh for the newly-created user. There is no chsh tool in DSM so you have to vim /etc/passwd and do it manually (with caution!).

For key-based SSH login, another trick is needed. If you use ssh-copy-id to copy your public key to the server, you still cannot log in because the weird umask settings in DSM makes everything of 777 permission. You need to chmod 700 ~; chmod 700 ~/.ssh; chmod 600 ~/.ssh/authorized_keys, or sshd would not let you in.

With these two tricks, a DSM is no different than a normal Linux box.

But DSM will reset /etc/paaswd file after restarting the system...

[noplanman]

@scruel For that you can define a scheduled task to run as "root" on "Boot-up".

My task looks like this, for the user backupper:

echo -e "$(cat /etc/passwd | grep -v ^backupper)\nbackupper:x:1032:100:Backup User:$(echo ~backupper):/bin/sh" > /etc/passwd

Of course you'll need to adapt to your needs.