[operator] Add backup #7441

lgarciaaco · 2019-12-18T11:03:44Z

Enable periodical and one time backups.

A backup contains syndesis openshift resources as well as a database backup. The backup packages allows for backups to be upload to an external datastore. At the moment, only S3 is supported, but I plan on adding rsh next.

Backing up from command line.

Backup

To perform a backup from command line just run syndesis-operator backup. This will create a backup folder with all backed up resources inside.

Uploading to external datastores is disabled with this mode.

You can set the output directory with --backup dir

syndesis-operator backup --backup /tmp/backup

Restore

To restore, simply run

syndesis-operator restore --backup /tmp/backup

restore also takes a zipped backup as an argument.

Running periodical backups

To enable periodical backups, fill up the backup section of the CR.

spec:
  backup:
    schedule: daily

schedule accepted values are: hourly;daily;midnight;weekly;monthly;yearly

schedule is a set once property, changing its value wont yield any result.

To disable periodical backups, leave in blank the backup section in the CR.

Enable S3 as external datastore

To enable S3 as external datastore you must create a secret as following:

kubectl create secret generic syndesis-backup-s3 \
  --from-literal=secret-access-key='replace with access key'  \
  --from-literal=secret-key-id='replace with key id' \
  --from-literal=region='replace with region' \
  --from-literal=bucket-name='replace with bucket name'

lgarciaaco · 2019-12-18T11:06:46Z

Fixes https://issues.redhat.com/browse/ENTESB-12437
Fixes https://issues.redhat.com/browse/ENTESB-12440

zregvart · 2019-12-18T11:11:53Z

Nicer commit messages?

heiko-braun · 2019-12-18T11:12:11Z

With this:

spec:
  backup:
    schedule: daily

Where does the backup go?

lgarciaaco · 2019-12-18T11:16:40Z

With this:
spec:
  backup:
    schedule: daily
Where does the backup go?

If you specify the secret for S3, to S3, if not, nowhere

zregvart · 2019-12-18T11:17:31Z

install/operator/pkg/syndesis/action/backup.go

+	"k8s.io/client-go/kubernetes"
+	"sigs.k8s.io/controller-runtime/pkg/manager"
+
+	cron "github.com/robfig/cron/v3"


Would it make more sense to have a k8s CronJob, instead of the backup running within the operator process?

no because I need to spin off a docker container for it

@lgarciaaco can you elaborate that? You can spin the same image as the one used by the operator with perhaps a different entrypoint/arguments?

Imho it is harder to work with pods than this way where the backup is tided up to the running binary. It is harder to develop and to debug. Why don't you tell me the benefits of doing it with a k8s cron job over using a cron withing the operator?

I see backup and configuration as two separate concerns, for me it makes more sense to have them separate as any issues with the backup will be limited to the backup CronJob, likewise any issues with the operator would not jeopardize the backup. CronJob also gives the operators more flexibility than a feature we develop on our own, and matching existing features in our codebase unnecessary brings on more code to maintain.

Those are arguable ...

as any issues with the backup will be limited to the backup CronJob, likewise any issues with the operator would not jeopardize the backup. CronJob

If the operator is jeopardize do I want to continue running backups? maybe syndesis is jeopardize all together? this seems to a bigger decision that requires some further discussion

also gives the operators more flexibility than a feature we develop on our own

Right now a feature I developed on my own allows me to do all I need. I don't need to use something different for the promise that in the future I might (or well might not) use some of it features. If I need those feature whatever it might be I can swap then, if the cron package I am using can't provide it.

Either way, my point is that using this library makes my development and test circle easier. I don't see any value using k8s cron over it, but if more engineers think it is better I will gladly change it

Those are arguable ...

I'm sorry, they're not, apart from the extend that you're willing to argue about them.

If the operator is failing because, say, it hits memory limits and gets OOMed, the backups are not run. Likewise if the backup ends up crashing the operator any pending configuration changes will not be applied.

Having the operator run the backups functionality that jeopardizes the backups and the rest of the operator functionality.

I'm okay with this getting merged to master and further improved upon, I'm not okay with this getting released in 1.9 as I think there will be issues in operating it.

I am not willing to argue about anything. As I said, if others see a value right now or If we figure we actually need it, I will gladly change it.

* master: (37 commits) refactor(ENTESB-12218): Fix missing camel-rest-openapi component in Camel BOMs refactor(ENTESB-12218): Use camel-rest-openapi components chore(test): exclude ServiceVdbGeneratorTest TEIIDTOOLS-854 improving error reporting on publish chore(build): setup maven-buildtime-extension chore: Update camel version to 2.23.2.fuse-760011 chore(jaeger-addon): Remove addon cluster directory chore: configure syndesisio/backport-action fix: Fixed nightly integration tests on CircleCi TEIIDTOOLS-890 changing publishing to a single service fix: removing usedby msg fix: changes in publishing heading fix: rework display of virtualization details header TEIIDTOOLS-879 Connects DV metrics page to backend TEIIDTOOLS-861 removing duplicate entries user doc: Added links to doc for extension mechanism and datavirt user doc: Add MS SQLServer entry to table of schema descriptions user doc: New doc that describes Schema specification in a database connection [ENTESB-12331] Adding auto-discovery for strimzi kafka/AMQ brokers ENTESB-12330: Bubble up custom-resource switch to syndesis install scripts ...

phantomjinx

LGTM.
May wanna squash the commits then add a single commit message?

phantomjinx · 2019-12-18T11:15:29Z

install/operator/pkg/apis/syndesis/v1alpha1/zz_generated.deepcopy.go

@@ -1,6 +1,6 @@
 // +build !ignore_autogenerated

-// Code generated by deepcopy-gen. DO NOT EDIT.
+// Code generated by operator-sdk. DO NOT EDIT.


This keeps alternating every time I do a commit. Any ideas why, eg. are we running different regenerations?

Yea it also bother me ... no idea why

pure-bot · 2019-12-18T12:01:11Z

Pull request approved by @phantomjinx - applying pr/approved label

pure-bot · 2019-12-18T12:01:11Z

Pull request approved by @phantomjinx - applying pr/approved label

lgarciaaco · 2019-12-18T12:03:58Z

LGTM.
May wanna squash the commits then add a single commit message?

Yea I was planning to do so as per Zoran comment

avano · 2019-12-18T13:11:10Z

I plan on adding rsh next.

rsh where?

lgarciaaco · 2019-12-18T13:28:42Z

I plan on adding rsh next.

rsh where?

Well I though it might bring some value if you can create a pod with a pvc and scp to it, as an alternative. The method would take the namespace, labels to select the pod, path to copy the backup over

avano · 2019-12-18T13:31:45Z

I plan on adding rsh next.

rsh where?

Well I though it might bring some value if you can create a pod with a pvc and scp to it, as an alternative. The method would take the namespace, labels to select the pod, path to copy the backup over

sure, I was just curious. Would this also be a part of upgrade process? Or will the backups be completely separate?

lgarciaaco · 2019-12-18T13:38:38Z

sure, I was just curious. Would this also be a part of upgrade process? Or will the backups be completely separate?

this is what I plan to use for the upgrade backup and rollback, and actually the rsh datastore to failover to the db pod(which has a pvc) in case there is no S3 enabled, as it will be the case for most of the people using 7.5.

[backport 1.9.x] Backport #7441 to 1.9.x

Luis Garcia Acosta added 3 commits December 11, 2019 14:08

backup

7e5b568

conflict

5f3153c

backup

ddad87b

pull-request-size bot added the size/XXL label Dec 18, 2019

lgarciaaco requested a review from phantomjinx December 18, 2019 11:03

pure-bot bot added the pr/review-requested Use this if you want to have a review. pure-bot will prevent merging if set and no review given label Dec 18, 2019

lgarciaaco requested a review from chirino December 18, 2019 11:04

lgarciaaco added group/operator Infrastructure operator related cat/enhancement Enhancements to existing features and capabilities cat/feature PR label for a new feature labels Dec 18, 2019

zregvart reviewed Dec 18, 2019

View reviewed changes

phantomjinx approved these changes Dec 18, 2019

View reviewed changes

pure-bot bot added the pr/approved Applied by pure-bot when a review is approved label Dec 18, 2019

pure-bot bot merged commit df9492d into syndesisio:master Dec 18, 2019

lgarciaaco mentioned this pull request Jan 9, 2020

[backport 1.9.x] Backport #7441 to 1.9.x #7516

Merged

phantomjinx added a commit that referenced this pull request Jan 9, 2020

Merge pull request #7516 from lgarciaaco/1.9.x

13f14ec

[backport 1.9.x] Backport #7441 to 1.9.x

lgarciaaco deleted the backup branch January 30, 2020 09:24

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[operator] Add backup #7441

[operator] Add backup #7441

lgarciaaco commented Dec 18, 2019

lgarciaaco commented Dec 18, 2019

zregvart commented Dec 18, 2019

heiko-braun commented Dec 18, 2019

lgarciaaco commented Dec 18, 2019

zregvart Dec 18, 2019

lgarciaaco Dec 18, 2019

zregvart Dec 18, 2019

lgarciaaco Dec 18, 2019

zregvart Dec 18, 2019

lgarciaaco Dec 18, 2019

zregvart Dec 18, 2019

lgarciaaco Dec 18, 2019 •

edited

phantomjinx left a comment

phantomjinx Dec 18, 2019

lgarciaaco Dec 18, 2019

pure-bot bot commented Dec 18, 2019

pure-bot bot commented Dec 18, 2019

lgarciaaco commented Dec 18, 2019

avano commented Dec 18, 2019

lgarciaaco commented Dec 18, 2019

avano commented Dec 18, 2019

lgarciaaco commented Dec 18, 2019

[operator] Add backup #7441

[operator] Add backup #7441

Conversation

lgarciaaco commented Dec 18, 2019

Enable periodical and one time backups.

Backing up from command line.

Backup

Restore

Running periodical backups

Enable S3 as external datastore

lgarciaaco commented Dec 18, 2019

zregvart commented Dec 18, 2019

heiko-braun commented Dec 18, 2019

lgarciaaco commented Dec 18, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lgarciaaco Dec 18, 2019 • edited

Choose a reason for hiding this comment

phantomjinx left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pure-bot bot commented Dec 18, 2019

pure-bot bot commented Dec 18, 2019

lgarciaaco commented Dec 18, 2019

avano commented Dec 18, 2019

lgarciaaco commented Dec 18, 2019

avano commented Dec 18, 2019

lgarciaaco commented Dec 18, 2019

lgarciaaco Dec 18, 2019 •

edited