Backups/Restore functionality

[p10tyr]

Is your feature request related to a problem? Please describe.
I have just started using nginxmanager quite a lot now with some production services too.
I did the easy setup with no external databases so I am wondering how to export/import?

Describe the solution you'd like
Export / Backup
Import / Restore

Describe alternatives you've considered
Don't know.. go into the docker container and hack into the sqllite db??

Additional context
Good for distater recovery too if we could just export data into a well known format and have it backup up away from the backup

[sanderlv]

I would LOVE to see this natively from within the user interface ---> hence that's the reason at all I like this product

[danielschenk]

I'm wondering: is atomicity always guaranteed on the sqlite file? In other words: can we be sure that the database on disk is never in an inconsistent state due to ongoing transactions?

If above can't be guaranteed, we need to always stop proxy-manager for a while to be able to pull a safe copy, which can be undesired. In that case, a function to have proxy-manager export a copy (on some kind of trigger which can be automated) would be very helpful. Home Assistant for example has a builtin service for this which can be hooked up to any automation.

[p10tyr]

If atomicity was a problem then SQLite probably wouldn't have been used? I don't think it has to be that over the top. The database only stores config and nothing about transactions so there is almost nothing happening in the databases. As an admin if you are exporting you wont be changing config at the same time you export either.

I was thinking a simple export to CSV function. I have not looked at the code and how it connects to the DB of choice but there should be a simple way to export map config and then reimport it on the other side.

I am not sure about the lets encrypt stuff. There are a lot of considerations here. I would just request a new certificate on the new host. problem solved. If you talking about HA.. or load balancing.. well thats a bigger problem that can be solved in other ways.

Looks like my request is pretty reasonable.. Sorry I haven't been back here for a while.. I mean.. I wouldn't mind looking at the code I guess.. why not!?

[danielschenk]

I'm also not familiar with the codebase at all. I just found your issue because I'm missing this kind of function myself too.

As can be read here atomicity is taken care of, but SQlite uses file locks from the OS for that. If we just read the SQLite file on our own (while proxy manager is also running) we can't be sure that blocks of the file are actually being written because we bypass SQLite's locking protocol. I've noticed that even without changing settings on my instance, the database file gets frequent updates, so I think it's too big of a risk to just read it while proxy manager is running.

Hence I think your feature request is very useful. Why a CSV and not just a copy of the SQLite file?

Regarding the certificates, I think just re-requesting them and not including them in a backup would indeed be the easiest way to solve this.

[TicTac9k1]

Usually you can do this with copying the data and letsencrypt directory of the compose folder, aka the whole npm docker compose folder.

I would, but copying or tarring gives me permissions errors I don't know for sure are relevant for a successful export, even as root.

https://i.imgur.com/tKfJ3BV.png

Using the jc21/nginx-proxy-manager:latest docker image through docker-compose.yml

[smaccona]

@danielschenk you are absolutely correct that copying the SQLite database file while the database is online is not safe from corruption. SQLite does have several built-in ways to backup a running database:

• The clone command, but this is slow and locks the entire database for writes while it's running (you can still read).
• The dump command, which dumps the schema and data as SQL to a text file. This is typically faster than clone, but still not "fast", and it also locks the database for writes.
• You can use vacuum into to defragment the database into another file. This is faster than the first two, but still not very fast and also locks the database for writes.
• The backup command, which is faster than all the others, and doesn't lock the whole database (it does perform locks, but it releases them and yields to other connections).
• There's also an API for performing backups (see https://sqlite.org/c3ref/backup_finish.html), but I don't know if it's supported in the sqlite3 package being used here (but you might be able to simply call the backup command by using exec() and passing .backup).