[Feature request] Backup from stdin

[agateblue]

Sometimes, simply copying a directory isn't enough for a backup. This happens for instance when you're backing up a SQL database such as MySQL or PostgreSQL: to ensure the backup integrity, you need to use pg_dump and backup the output of that command.

Luckily, restic supports backing up data piped via stdin. Do you think it would be possible to support this in autorestic?

What I currently have in mind is:

locations:
  psqldata:
    command: pg_dumpall

Which autorestic would internally translate to pg_dumpall | restic backup --stdin --stdin-filename psqldata.

I'm going to experiment with that, let me know what you think about it :)

[cupcakearmy]

Same with mongodump, might be usefull yes. Altouhgh at least with mongo it's not seen as the reccomended way to go about it.

[bthj]

@EliotBerriot , your backup journey article took me here and I'm subscribing to this issue :)

At the risk of hijacking the discussion here, I'd like to ask if you're currently using your own tool, savior, in conjunction with autorestic?

[cupcakearmy]

I will do some testing this WE (hopefully I don't forget xD)
might be a bit tricky piping it through node and then to restic, but we'll see.

[agateblue]

At the risk of hijacking the discussion here, I'd like to ask if you're currently using your own tool, savior, in conjunction with autorestic?

@bthj no I've completely dropped it, I've been using https://github.com/cupcakearmy/autorestic#before--after-hooks (dump in a before hook, remove the dump in the after hook, backup the dump) as a workaround until we can pipe backups directly to stdin.

I will do some testing this WE (hopefully I don't forget xD)
might be a bit tricky piping it through node and then to restic, but we'll see.

@cupcakearmy yeah, wy first attempts at this (back in december) got stuck at this point unfortunately: piping stdin to a subprocess launched by node.

[cupcakearmy]

As this feature is not really relevant anymore and adds significant implementation I will close this. feel free to reopen if necessary

[berlincount]

I'd prefer to have this reopened.

I'm essentially forced to implement my backup solution without autorestic because dumping a large database to disk inbetween is simply prohibitive :-/

[xpufx]

@berlincount A similar tool 'resticprofile' supports backing up from STDIN. I actually have a different problem with it regarding hooks that is not supported and hoped autorestic would fix this situation.

[nser77]

Many thanks to @agateblue for raising that.

I personally agree with @berlincount: using the hooks script will create an intermediary file which will increase disk space and decrease security.

Has someone encountered a solution or even a valid alternative?