S3 backup #107
S3 backup #107
Conversation
|
I think this approach LGTM, I like that it is using the environment variables via a secret. Should maybe add optional resources for the backup container pod so that it does not steal cycles from the nats-streaming server one during the backup process? Also would be good to add a usage example in the README from the chart before merging. Thanks for the contribution! |
…` to a slightly more reasonable default
…dds basic example values
| args: | ||
| - | | ||
| while true; do | ||
| curl -s 127.0.0.1:8222/streaming/serverz | grep 'Leader' |
There was a problem hiding this comment.
Talking to @kozlovic on whether only backup the Leader, and we now think that it might be simpler and safer to backup leader and follower nodes as well into different folders using the $POD_NAME, something like:
aws s3 cp {{ .Values.store.volume.mount }}/* {{ .Values.store.backup.s3Path }}/$POD_NAME
There was a problem hiding this comment.
Right. Backup each node regardless of their leadership status. This would prevent issues when leadership changes and file stores are not necessarily yet at the same stage.
There was a problem hiding this comment.
Ok, just removed the check for Leader, and also added a prefix using stan.name/POD_NAME to make sure different deployments of stan could use the same bucket without collisions.
This is an initial, sort of naive implementation to allow backing up the stan file store. I'm looking for some feedback on approach here. Addresses #104 and #106