Integrate wrestic into k8up #474
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This is a wraper for Restic that is intended for use in a docker container. It's main use is for the baas project, but it can easily be used for standalone deployments. Relates to: VSHNOPS-1096
This commit contains various improvements concerning the error detection and various bugfixes. It now detects wrong credentials and prints an error.
This change adds a timeout for the snapshot listing. So it is possible to detect if the repository is reachable or not.
With this change it is possible to do backups with the openshift stdout as the source. For this to work properly most of the generic command section and the output handling had to be rewritten. Now the outputs of the commands are printed in realtime as they happen. Also the stdout and stderr of the commands are handled seperately. This makes detecting errors much easier. To determine if/how many errors there were the lines from stderr are counted since restic only prints errors to stderr.
If the shell form of ENTRYPOINT is used in the dockerfile it loses the arguments due to escaping. But we do need a shell for this to work so now there's a wrapperscript that starts a shell and passes the args correctly.
The triggered metric updates are now no more in their own thread. This simplifies waiting for the requests to finish quite a lot.
Support for backup commands and consistent database backups See merge request vshn/wrestic!4
To update Prometheus metrics with the current snapshot count.
List snapshots after backup or prune See merge request vshn/wrestic!5
Dev See merge request vshn/wrestic!6
With this commit the wrestic container posts it's metrics and a list of available snapshots to an arbitrary HTTP endpoint. Relates to: AMZE-995
Push stats to http endpoint See merge request vshn/wrestic!7
This commit implements restores to local disk and to a s3 bucket. Please see the readme on how to use them. Relates to: AMZE-996
If if `cmd.Wait()` was reached before all output was processed it was possible to encounter a "File already closed" error. This is due to `cmd.Wait()` closing the pipes of the command. By moving the output locks to before `cmd.Wait()` this issue gets resolved because the output processing is guaranteed to be complete at that point.
Feature/restore See merge request vshn/wrestic!8
Webhook after S3 restore See merge request vshn/wrestic!9
Feature/pv csinwebhook See merge request vshn/wrestic!10
Archive function See merge request vshn/wrestic!11
This commit enables wrestic to send a list of snapshots to the webhook endpoint after every command that could change the repository. These include: * prune * backup * stdin backup Relates to: AMZE-1117
With this commit there will be a snapshot for every PVC mounted (or every folder under /data to be more precise). So for every folder there will be a separate snapshot and webhook with stats. Relates to: AMZE-1077
Feature/webhookafterprune See merge request vshn/wrestic!12
Make a snapshot per mounted PVC See merge request vshn/wrestic!13
This was referenced Aug 16, 2021
Because `make build` will always build a GOOS=linux binary, this can be used to run k8up also on Mac and probably Win.
simu
approved these changes
Aug 16, 2021
ccremer
approved these changes
Aug 16, 2021
Co-authored-by: Chris <github.account@chrigel.net>
Co-authored-by: Chris <github.account@chrigel.net>
Kidswiss
approved these changes
Aug 20, 2021
This was referenced Aug 20, 2021
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Integrates wrestic into the k8up repository and unifies them into a single binary.
This is the bulk-load of changes required to finish #359 eventually.
Tasklist:
unification of configuration methodsSee these instead:docsSee these instead:Checklist
Keep pull requests small so they can be easily reviewed.I tried my best, but that's really hard here.bug,enhancement,documentation,change,breaking,as they show up in the changelog