Remove antiquated msmtp and ntfy images

[jetchirag]

Following the discussion on #130 and #144, I've made the necessary changes to discontinue the msmtp and ntfy flavors of the Docker image in this pull request.

I've also moved the base flavor to top level of the repo.

[witten]

I'm not really a dev on this project, but just chiming in: This looks great to me!

[m3nu]

Just tried this from building to using and I think it's a great start. (I never used this image before). Here my thoughts and ideas. I know that you took some of this from the previous version, so this is just general discussion, not this PR specifically. Some things are also more ideas and if there are arguments against them, we should discuss them.

README

• The section 'Starting and stopping containers from hooks' is a very special use case that should be further down with examples on how to mount the Docker socket.
• "Example run command": This is the most important part. How can I quickly set up my backups. Since this needs some folders and setup outside the container, we should include those commands too. E.g.
  • Create required folders: `mkdir -p ./data/{borgmatic.d,.cache,.config,...}
  • Adjust or copy borgmatic.yaml
  • Init repository
• Environment: All supported env vars should be listed (as table?) with default value and if they are required.

README structure

High level sections in the README could be:

1. Description: What this does? Run Borg and Borgmatic in a container. Optionally the Cron daemon.
2. Usage: Different ways to use this image
   1. The minimal shell commands to run Borgmatic as binary only. Similar to Example run command. mkdir ...; docker run ... borgmatic init...
   2. Running as daemon: Still using docker run here, but keep the it running for continuous backups
   3. Running with docker compose. For more structured deployment. Links to the docker-compose.yml file and .env.template file and shows how to adjust.
3. Volumes: Explain which folders can be mounted. Best as table
4. Environment Variables: Explain which env vars can be passt. Best as table. Should mention that all Borg/Borgmatic env vars will also work and link to them.
5. Additional Uses

• Stopping containers with hooks/including Docker socket
• FUSE

Usage as binary or daemon

It took me a while to figure out that this image will by default start a Cron daemon and not do anything for a while. I feel like using this image as borgmatic binary is a second use case to cover. E.g. as described here.

Question is if we let users use their own Cron (like a Systemd timer) or manual backups only. I feel forcing this cron daemon is limiting the usefulness. Both uses, as daemon and as binary are useful.

How about explaining this Cron thing better and adjusting the entryscript to allow usage as binary IF a subcommand is provided. E.g. this will run borgmatic list:

$ docker run \
  --rm -it \
  -v /home/manu/Workspace:/mnt/source:ro,Z \
  -v ./data/borgmatic.d:/etc/borgmatic.d/:Z \
  -v ./data/.config/borg:/root/.config/borg:Z \
  -v ~/.ssh:/root/.ssh:Z \
  ghcr.io/borgmatic-collective/borgmatic \
  list

Issue here is of course that we have so many mounts to pass each time, so the command is a bit long. I would still offer this case as example.

The entryscript would need to be adjusted to check for commands to be specified. If none are set, run the Cron daemon. If they are set and aren't bash or sh, run them as borgmatic subcommand?

Volumes

• Never seen so many volumes. Hope this can be simplified.
• borgmatic.d will only contain a single borgmatic.yaml file? So why not mount only this file so it's easier to work with on the host?
• VOLUME_BORGMATIC_STATE var is missing in the .env.template file. Some of those volumes can be merged maybe to simplify it?

Building

• .github/workflows/build_*: Those files look almost the same. Sure we need both?
• Dockerfile, COPY --link requirements.txt /. I removed --link here since it's buildkit specific and didn't work with Podman which I use.
• I was never able to stop the container gracefully. Seems like this cron software needs SIGKILL

[m3nu]

(Since my suggestion mostly restructures the README, it could be a separate PR)

[jetchirag]

@m3nu Thank you for all the suggestions! I'm writing a new readme file, section by section, but I expect it to take some time since I need to understand some of the points myself so I can better explain them in the guide.

For VOLUME_BORGMATIC_STATE, see #197

As for the PR, I'm thinking if I should use this one for these changes?

[m3nu]

Thanks for taking this into account. I think it will greatly help people use this role. That's based on my experience with running a Docker hosting service and integrating about 60 different apps and the support provided for BorgBase.

If you want, you can adjust this PR. Since it's limited to the README. For other things, maybe we add a new PR? E.g. making it easier to use the image as binary.

Also, for the next milestone, I don't see too much to do. The image is already quite slim now. So making it multi-stage and removing a few deps will be quick. We can spend the other time on usability.

[modem7]

Hey guys - thought I'd post quickly as I'm not ignoring this.

Work is slamming me, will try and get a review done on the weekend if that's alright!

[jetchirag]

@m3nu I've pushed just changes entry file changes. Can you check if that's what you were referring to? I'll explain the usage in Readme.

I was also wondering if that's how images are generally built since I haven't much many. Since it could be achieved before by passing whole command borgmatic list instead of just subcommand. And if there are any cons of this approach.

[jetchirag]

I was never able to stop the container gracefully. Seems like this cron software needs SIGKILL

Took me some time to figure this out. It's happening because supercronic is executed by shell which ignores SIGTERM. If I switch entry.sh for directly running supercronic, it's resolved.

Since we want to use the script, we can add exec before supercronic to switch main process to supercronic command which docker can monitor.

---

Dockerfile, COPY --link requirements.txt /. I removed --link here since it's buildkit specific and didn't work with Podman which I use.

What do you suggest? Podman seems to have added support for buildkit.

---

borgmatic.d will only contain a single borgmatic.yaml file? So why not mount only this file so it's easier to work with on the host?

Seems to contain two files, config.yaml and crontab.txt

[m3nu]

exec sounds like a good solution then. I'm sure other projects have already solved this issue too.

I'd keep Dockerfiles simple to avoid being dependent on specific versions. Already tried buildkit in podman, but it didn't work with e.g. docker-compose.

crontab.txt is only for this image. It's not from Borgmatic. Could the cron schedule be an env var?

[jetchirag]

Makes sense. I'll remove --link.

crontab.txt is only for this image. It's not from Borgmatic. Could the cron schedule be an env var?

It can be. I can have it create cronjob file in entry.sh with passed variable or default to 0 1 * * *. Follow up question, is once a day enough for default value?

[m3nu]

Yes, once a day is fine as default. Daily backups are most common. Especially with more data, like a few TB.

[m3nu]

Was thinking about the "as binary" thing a bit more: Maybe it's not too clear what happens, if we treat some commands as Borgmatic commands and somehow redirect them.

Check out this Restic Docker setup: https://github.com/djmaze/resticker

They have some good concepts:

• I like how they use env vars for Cron
• And how they show running any command via docker-compose (here)
• Since we have many volumes, having this config in Docker-compose makes sense. But README should also show how to do it without.
• I like the RUN_ON_STARTUP option to do a backup right away.
• They also use exec as your suggested. And the entryscript knows about the subcommand.

[rallisf1]

borgmatic.d will only contain a single borgmatic.yaml file? So why not mount only this file so it's easier to work with on the host?

Seems to contain two files, config.yaml and crontab.txt

I have tons of files in borgmatic.d as I am backing up multiple projects in the same repo by using archive_name_format

[m3nu]

Noted, then let's see if any other mounts could be optional or grouped.

[jetchirag]

And how they show running any command via docker-compose

Wrote about that [here](And how they show running any command via docker-compose). I did however run into a problem I couldn't find a fix to.

Since we have many volumes, having this config in Docker-compose makes sense. But README should also show how to do it without

I added a table previously. Do we need anything else?

They also use exec as your suggested. And the entryscript knows about the subcommand.

It seems to only know about unlock command otherwise it's also executing exec restic "$@" as my earlier commit. It is however verifying the repo. Should we add something like that too?

Also added the other two variables you mentioned.

[modem7]

From what I can see, this looks good - I have not tested the code changes, however.

[witten]

I did a merge from master.

[jetchirag]

Thank you for solving merge conflicts.

I couldn't find a solution to this. Can someone take a look?

docker-borgmatic/README.md

Lines 170 to 174 in 638c2d6

	If you want to run borgmatic commands using this configuration instead of starting the container as daemon, you can run:
	<!-- TODO: entry.sh is not working with docker-compose, having to pass full command -->
	```
	docker-compose run --rm borgmatic borgmatic list
	```