add new plugin to backup using git

[AdSchellevis]

In order to gain traceability between configuration changes a backup option using git would be a practical addition.
From OPNsense the supported workflow should implement the following pattern:

As a requirement for this functionality we need to be able to subscribe on config changes, which is a feature requests in core (opnsense/core#4388)

Planned supported functionality

The plugin should include the following functionality:

• User configurable via the web interface within the same structure as Google backup and Nextcloud
• User/pass (over https) or private key (ssh) authentication
• Configurable upstream repository and branch selection (which need to be "bare" on the first backup)

Conflict resolution:

Since the initial backup needs to set an upstream, we need to trust the upstream repository (branch) is either bare or in sync with our local situation. If that's not the case, we need to record this and (when using the test button) report to the user we can only push to a new (bare) repository. (no fast-forward or merge scenario's will be supported from the user interfaces)

Sensitive data

Git backups will be unencrypted, in order to compare differences properly, with (highly) sensitive data we need to make sure we signal the user accordingly to not use public infrastructure for these types of backups.

sponsored by : Modirum (https://www.modirum.com/)

[mimugmail]

Nice feature and sponsoring :)

[carpenike]

Very nice!

Any chance of integrating with git-crypt for sensitive information or would that require the config backend to also be able to reference secrets in a file separate from config.xml?

[AdSchellevis]

@carpenike git-crypt isn't in scope of this project. I haven't looked at git-crypt in much detail, but wouldn't this mean all generated diffs would be quite large since comparing crypted data? It's not very complex to encrypt the config.xml (as the other backup providers offer), but it would defeat the purpose of easily being able to track changes (all data in the backup is sensitive by nature in this case). You can probably just add git-crypt manually (it looks like a filter, which shouldn't change the regular workflow)

[carpenike]

The diffs should be similar in size I believe, but the only way to see the details would be on a client that can decrypt them. So to your point it would eliminate the value of having a web based git tool from being able to show changes.

[AdSchellevis]

I don't think thats theoretically possible.

[AdSchellevis]

test instructions on 20.7.3:

opnsense-patch 5ea5f94b
service configd restart
pluginctl -s syslog-ng restart

Next test the generic handler, change something in the configuration (hit save no a random page) and check if cat /conf/event_config_changed.json exists.

Next install the plugin (use tools https://github.com/opnsense/tools to checkout the latests sources) and execute the following:

cd /usr/plugins
git checkout master
git pull
cd sysutils/git-backup
make upgrade

Go to System->Configuration->Backups and configure your git upstream repo. "Setup/Test" should flush the changes to upstream.

[mimugmail]

I tested it right now, works really nice!

[AdSchellevis]

@mimugmail thanks! good to hear :)

[qdrop17]

nice plugin @AdSchellevis!

I just posted something in the forum: https://forum.opnsense.org/index.php?topic=19674
It seems like you guys are still implementing that feature.

Regarding the git-crypt: Maybe it makes sense to be able to replace certain strings before committing. Reason for that: We use Wireguard as our VPN technology and the go-wireguard plugin stores the private keys inside the config.

I might also be able to simply sed them inside your script anyways.

[AdSchellevis]

@qdrop17 documentation is a bit ahead of time indeed.

Regarding the git-crypt: Maybe it makes sense to be able to replace certain strings before committing. Reason for that: We use Wireguard as our VPN technology and the go-wireguard plugin stores the private keys inside the config.

I don't agree, string replacement is quite dangerous and leads to an endless path of trying to figure out what is sensitive (which might worst case vary from installation to installation).

For us this ticket is closes, we don't have other changes planned ahead (other than moving the plugin to release)