Conversation
JamesDoingStuff
requested review from
TBThomas56,
davehadley and
iamvigneshwars
There was a problem hiding this comment.
why do we have a staging being backed up and not prod?
There was a problem hiding this comment.
Probably best to roll it out on staging first and just make sure all is well - I just needed to add something to the Values.yaml for prod
There was a problem hiding this comment.
Makes sense! I would add a simple ticket to the backlog in case it has not been added yet
| namespace: workflows | ||
| type: Opaque | ||
| {{ else }} | ||
| {{- end }} |
There was a problem hiding this comment.
should there be an empty line after all of these?
There was a problem hiding this comment.
Does it do anything? I see that the other templates have one, so I don't mind adding one in if it's convention, just curious
charts/workflows-cluster/templates/etcd-rclone-scripts-configmap.yaml
Outdated
Show resolved
Hide resolved
JamesDoingStuff
force-pushed
the
jg/etcd-backup
branch
from
8abd3b1 to
ceddb28
Compare
|
|
||
| # Delete old backed up objects, with age >= 2 days. | ||
| echo "deleting old backups from echo s3" | ||
| rclone delete --min-age=2d echo:dls-workflows-prod/${PREFIX} |
There was a problem hiding this comment.
this is fine for this PR but we should decide our strategy for how many and how long we want to keep backups for.
JamesDoingStuff
force-pushed
the
jg/etcd-backup
branch
from
e730483 to
4d2e13b
Compare
JamesDoingStuff
force-pushed
the
jg/etcd-backup
branch
from
433f38b to
3beb939
Compare
|
Made a couple of minor changes to get this passing CI (last 2 commits) so if someone could just sanity check those please, that'd be great :) |
3 participants
The
dev_resources/buildCI is currently failing due, I think, tocheck_k8s_resources.pynot handling CronJobs well - specifically, having aresourcesfield but noreplicas. I'll look into fixing thisAdds:
dls-workflows-prod/<staging/prod>/etcd-snapshot-<timestamp>.db. The contents are encrypted. The job deletes backups older than 2 days.etcdctl snapshot restoreon the provided etcd volume. This job won't automatically.scripts/restore-etcd.sh) that scales down the etcd and the vcluster, performs the above job for each etcd volume, then returns the cluster to initial levels.