Adds docker image. #4372

pierreozoux · 2019-02-15T20:33:11Z

Relates to #4364

This is the first step, and then once you detail what is the release process, we can add that later.

It always build latest, but if you want we can add an ARG with the version.

ThomasWaldmann · 2019-02-16T09:31:33Z

Dockerfile

@@ -0,0 +1,20 @@
+FROM debian:stretch


docker-library/official-images#1516

ends with:

So for most of us there is no way to run verified trusted docker images in production. Very unfortunate.

I am not familiar with docker, but sounds pretty bad?

You're not wrong, but in order for an attack like this to happen a malicious user would have to:

Steal the Docker corporation's login credentials

Point the Docker library repo from the official debian repo to somewhere else (or compromise the Debian repo)

Put it up for review and hope nobody notices the alarm bells that will be going off everywhere

Wait for the automated build to push the official image up (yet again that would throw a fit)

Clean any cached images on a user's disks

Re-pull and re-run images

All content (official and not) served from Docker is done so via HTTPS/TLS so stealing credentials should be the only vector of attack.

well, my idea of security is not to rely on some infrastructure being secure and nothing going wrong, but to just check a gpg signature. it doesn't matter how or from where you get the stuff as long as you can verify the sig.

You could pin to a sha256 hash of the base image instead of a tag, but then you'd have the maintenance burden of verifying every new version manually and updating this hash.

Gpg is no magic security fairy dust either, in Debian you also rely on the infrastructure of whoever created and distributes the signature. Unfortunately Docker images are not created in a deterministic way themselves (and this Dockerfile also adds additional nondeterminism on top by using whatever package version is current at build time).

@MarkusTeufelberger if you have the correct public key of the release maintainer, you can verify all releases with their signature, no matter how you get the release or the release signature. you can not do that just with sha256.

There is a new feature Docker content trust, I'm not familiar with it yet, and this is an "enterprise feature" (at least in docker)... But other container runtime use it also (like cri-o) and other docker registry too (like quay.io). I propose that we investigate that in a future iteration.

Dockerfile

ThomasWaldmann · 2019-02-16T09:50:52Z

one general thing: the whole docker image building should run inside the stretch64 vagrant vm.

DarrienG · 2019-02-17T15:13:25Z

I vote for versioned building. We're asking for trouble always building the latest.

ThomasWaldmann · 2019-02-22T16:47:02Z

Maybe it could be also moved to scripts/Dockerfile? There are already too many files in ..

ThomasWaldmann · 2019-03-09T14:40:18Z

@pierreozoux are you still working on this?

codecov-io · 2019-03-10T20:07:22Z

Codecov Report

Merging #4372 into master will increase coverage by 0.67%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #4372      +/-   ##
==========================================
+ Coverage   83.71%   84.38%   +0.67%     
==========================================
  Files          37       37              
  Lines        9567     9826     +259     
  Branches     1596     1611      +15     
==========================================
+ Hits         8009     8292     +283     
+ Misses       1090     1074      -16     
+ Partials      468      460       -8

Impacted Files	Coverage Δ
src/borg/archive.py	`82.96% <0%> (+0.63%)`	⬆️
src/borg/archiver.py	`83.72% <0%> (+3.02%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 85b010a...4396a8d. Read the comment docs.

pierreozoux · 2019-03-10T20:22:38Z

Sorry for the delay, holiday.. ;)

I put some brain juice on it. And about your questions, I think this PR is the best possible way to tackle that.

Don't install third party in the VM

docker is a third party repo in debian.

Copying from VM to local machine would have been a hassle, so I though it is just pain easier to build it from your dev machine. (as anyway you'll need the client for pushing to the hub).

use `scripts` folder for Dockerfile

I think I like that the Dockerfile is at the root. As a docker user, in a quick look, I can see docker support of a repo, but no strong opinion there.
If you run the build_and_publish script from the root of the repo, all should work.
You'd need a docker hub account thought. I created the org already, once you have your docker handle, give it to me, and I make you admin (and then you can remove me, I'm fine).

pip vs setup.py vs pyinstaller

I could have choosen another method, but I'm not really familiar with python. And so I though that the fat binary was equivalent to any other method. But I have no strong opinion there. If you have a strong opinion, we can change that.

borgbackup and borgserver

It is just an attempt, if it creates too much debate, I'd prefer to not ship it with this PR, and open a separate PR

ThomasWaldmann

some stuff i found.

scripts/docker/README.md

scripts/docker/build_and_publish.sh

scripts/docker/borgbackup

scripts/docker/README.md

DarrienG · 2019-03-12T14:10:39Z

scripts/docker/README.md

+  --entrypoint=/usr/bin/borgbackup \
+  --name borg-backup \
+  borgbackup/borg
+docker rm borg-backup


It probably makes sense to just run with docker run --rm so you don't have to run docker rm afterwards. Nothing is saved inside the container (which is good), so it's best to be a good citizen and remove it afterwards.

it is using docker volumes, so it would delete the associated volume.

DarrienG · 2019-03-12T14:16:21Z

scripts/docker/README.md

+
+```
+docker pull borgbackup/borg
+docker run \


As long as you use json syntax (which we are) for the entrypoint, you don't need to do anything weird to pass arguments to the container.

You could use docker run --rm borgbackup/borg -v $YOUR_MOUNTS $NORMAL_BORG_OPTIONS as if it were a binary.

dargmuesli · 2019-03-16T18:04:37Z

Let me add a quick thank you for working on this guys! :)

pierreozoux · 2019-03-31T08:20:35Z

Ok, I separated the documentation from the binary and the convenience scripts (borgbackup and borgserver), I clearly tagged them as beta.
I think we are now good to get started like this, and improve in an iterative manner.
(I'll also use the convenience script in my production, once I use them, I'll probably add some fix and remove the beta flag :) )

dargmuesli · 2019-03-31T22:27:26Z

scripts/docker/CLIENT.md

@@ -0,0 +1,177 @@
+## CLient side


pierreozoux · 2020-08-25T14:53:10Z

@ThomasWaldmann if I remove all the client and server scripts, would you feel better about it?

I can spend a bit of time to rework this to have a basic docker image at least.

I'll be able to maintain it as well, as I'm still using this.

ThomasWaldmann · 2020-08-25T19:07:14Z

Yeah, would feel better with a very basic thing. But as I don't use docker, I can't comment much about how that should look like.

But there were some other reviewers here, maybe you can discuss about how a basic docker image for borg should look like, so that it is generally useful.

Nielio · 2020-12-22T11:31:11Z

Any updates on this? Would be nice to have an official containerd borg !

pascaldornfeld · 2021-06-18T18:14:53Z

I am looking for an official image too

williamdes · 2021-07-07T12:57:17Z

Dockerfile

@@ -0,0 +1,9 @@
+FROM debian:stretch


No sure this program needs Debian
You should use Alpine

Docker content trust works fine, you can read how to automate or use it here: https://github.com/sudo-bot/action-docker-sign

rugk · 2021-08-24T22:15:26Z

scripts/docker/borgserver

+    -e 's/^#PasswordAuthentication yes$/PasswordAuthentication no/g' \
+    -e 's/^PermitRootLogin without-password$/PermitRootLogin no/g' \
+    /etc/ssh/sshd_config
+  if [[ $(ls /etc/ssh-keys/ssh_host_*) ]]; then


AFAIK one should not use ls in scripts/parse it. Okay, maybe there is not much parsing happening, but I you may anyway get crazy edge-cases/it is just not clean when you e.g. have spaces or so in there…
What I wanna say: Maybe there is a better solution? 🙂

Also, were these scripts tested with shellcheck? (Just pasted this script and it did not complain to my surprise, so yeah,hmm…)

rugk · 2021-08-24T22:17:30Z

scripts/docker/README.md

+from the borg command line to continue this Quick Start.
+
+Note that this example is using the `latest` version of borg, and we recommend you
+to pin the version by appending version e.g. `borgbackup/borg:1.0.7` to use `1.0.7`.


Hmm why would you recommend it?

I mean, maybe to not break setups the major version could be pinned aka borgbackup/borg:1, but semantic versioning says everything else should not break the API…

rugk · 2021-08-24T22:19:21Z

scripts/docker/SERVER.md

+
+To start a borg server:
+ - generate an ssh-key
+   - `ssh-keygen -f ./id_rsa -N '' -t rsa`


It's 2021, not sure whether recommending RSA keys is still what we need. ED25519 could be used just as that, is shorter and at least as secure…

rugk · 2021-08-24T22:24:37Z

scripts/docker/borgserver

+  chmod 400 /etc/ssh-keys/ssh_host_*
+  sed -i 's|/ssh/|/ssh-keys/|' /etc/ssh/sshd_config
+  sed -i '/^#HostKey/s/^#//' /etc/ssh/sshd_config
+  sed -e "s#SSH_KEY#${SSH_KEY}#g" /home/borg/authorized_keys.sample > /home/borg/.ssh/authorized_keys


Hmm, where does this sample file come from? Debian I guess?
'
Okay, then it is empty.

Anyway, you should really suggest to save this file/dir in some (persistent) volume or prefill it (and not only SSHFS_IDENTITY_FILE), because otherwise you can get client connection errors…
Or oh well, this is the server part, nevermind I comment above…

But should not this be configurable via a command-line, too? Otherwise, clients cannot connect if they are not authorized.

Also you'd like to restrict the commands to borg serve for security reasons.
See https://borgbackup.readthedocs.io/en/stable/deployment/central-backup-server.html#restrictions and https://borgbackup.readthedocs.io/en/stable/deployment/hosting-repositories.html

The script here could do that too/configure it properly…

rugk · 2021-08-24T22:28:34Z

scripts/docker/borgbackup

+if [ -n "${SSHFS:-}" ]; then
+  if [ -n "${SSHFS_IDENTITY_FILE:-}" ]; then
+    if [ ! -f "$SSHFS_IDENTITY_FILE" ] && [ -n "${SSHFS_GEN_IDENTITY_FILE:-}" ]; then
+      ssh-keygen -t rsa -b 4096 -N '' -f "$SSHFS_IDENTITY_FILE"


Again in 2021, we maybe should use ed25519 instead of RSA…

rugk · 2021-08-24T22:31:56Z

scripts/docker/borgbackup

+    SSHFS_PASSWORD_OPT=''
+  fi
+  mkdir -p /mnt/sshfs
+  eval "${SSHFS_PASSWORD} sshfs -o StrictHostKeyChecking=no ${SSHFS} /mnt/sshfs ${SSHFS_IDENTITY_FILE} ${SSHFS_PASSWORD_OPT}"


StrictHostKeyChecking=no? Please, by all means, do not do this. This image is likely intended to be used as a client with remote servers, so not checking the server key could allow MITM attacks…

So better pass in the host key either as a variable or just use SSH's default TOFU security scheme, at least – with allowing the user to confirm the step/check the key manually. After all, they will run this manually one time at least, so that should work.

pierreozoux · 2021-08-25T08:29:22Z

Sorry, I'll close this PR, feel free to make your own PR based on this code or not.
(I'm not using borg any more)

It was a really nice software, I really enjoyed it! We are now using s3 and minio, using versionning and mirroring of s3, basically our requirements changed from let's have our backups encrypted to let's have a mirror site that can start fast in case of emergency.

ThomasWaldmann reviewed Feb 16, 2019

View reviewed changes

Dockerfile Outdated Show resolved Hide resolved

ThomasWaldmann reviewed Feb 16, 2019

View reviewed changes

Dockerfile Outdated Show resolved Hide resolved

ThomasWaldmann reviewed Feb 16, 2019

View reviewed changes

Dockerfile Outdated Show resolved Hide resolved

pierreozoux force-pushed the docker-image branch from 9091277 to 2ae2bcf Compare March 10, 2019 20:07

pierreozoux mentioned this pull request Mar 10, 2019

Official docker image pschiffe/docker-borg#7

Closed

ThomasWaldmann reviewed Mar 10, 2019

View reviewed changes

DarrienG reviewed Mar 12, 2019

View reviewed changes

Adds docker image and associated scripts.

4396a8d

pierreozoux force-pushed the docker-image branch from 2ae2bcf to 4396a8d Compare March 31, 2019 08:16

dargmuesli reviewed Mar 31, 2019

View reviewed changes

scripts/docker/CLIENT.md

@@ -0,0 +1,177 @@

## CLient side

Copy link

dargmuesli Mar 31, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Client

williamdes reacted with thumbs up emoji

ThomasWaldmann mentioned this pull request Aug 5, 2020

Official Docker image? #4364

Open

williamdes reviewed Jul 7, 2021

View reviewed changes

rugk mentioned this pull request Aug 24, 2021

Docker image rugk/borg-cron-helper#80

Open

rugk suggested changes Aug 24, 2021

View reviewed changes

rugk mentioned this pull request Aug 24, 2021

Script comments pschiffe/docker-borg#10

Closed

pierreozoux closed this Aug 25, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adds docker image. #4372

Adds docker image. #4372

pierreozoux commented Feb 15, 2019

ThomasWaldmann Feb 16, 2019 •

edited

Loading

DarrienG Feb 17, 2019

ThomasWaldmann Feb 17, 2019

MarkusTeufelberger Mar 6, 2019

ThomasWaldmann Mar 9, 2019

pierreozoux Mar 10, 2019

ThomasWaldmann commented Feb 16, 2019

DarrienG commented Feb 17, 2019

ThomasWaldmann commented Feb 22, 2019

ThomasWaldmann commented Mar 9, 2019

codecov-io commented Mar 10, 2019 •

edited

Loading

pierreozoux commented Mar 10, 2019

ThomasWaldmann left a comment

DarrienG Mar 12, 2019 •

edited

Loading

pierreozoux Mar 31, 2019

DarrienG Mar 12, 2019 •

edited

Loading

dargmuesli commented Mar 16, 2019

pierreozoux commented Mar 31, 2019

dargmuesli Mar 31, 2019

pierreozoux commented Aug 25, 2020 •

edited

Loading

ThomasWaldmann commented Aug 25, 2020

Nielio commented Dec 22, 2020

pascaldornfeld commented Jun 18, 2021

williamdes Jul 7, 2021

rugk Aug 24, 2021

rugk Aug 24, 2021

rugk Aug 24, 2021

rugk Aug 24, 2021

rugk Aug 24, 2021

rugk Aug 24, 2021

rugk Aug 24, 2021

pierreozoux commented Aug 25, 2021

Adds docker image. #4372

Adds docker image. #4372

Conversation

pierreozoux commented Feb 15, 2019

ThomasWaldmann Feb 16, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ThomasWaldmann commented Feb 16, 2019

DarrienG commented Feb 17, 2019

ThomasWaldmann commented Feb 22, 2019

ThomasWaldmann commented Mar 9, 2019

codecov-io commented Mar 10, 2019 • edited Loading

Codecov Report

pierreozoux commented Mar 10, 2019

ThomasWaldmann left a comment

Choose a reason for hiding this comment

DarrienG Mar 12, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

DarrienG Mar 12, 2019 • edited Loading

Choose a reason for hiding this comment

dargmuesli commented Mar 16, 2019

pierreozoux commented Mar 31, 2019

Choose a reason for hiding this comment

pierreozoux commented Aug 25, 2020 • edited Loading

ThomasWaldmann commented Aug 25, 2020

Nielio commented Dec 22, 2020

pascaldornfeld commented Jun 18, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pierreozoux commented Aug 25, 2021

ThomasWaldmann Feb 16, 2019 •

edited

Loading

codecov-io commented Mar 10, 2019 •

edited

Loading

DarrienG Mar 12, 2019 •

edited

Loading

DarrienG Mar 12, 2019 •

edited

Loading

pierreozoux commented Aug 25, 2020 •

edited

Loading