Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: Add backup and restore commands to the makefile #107

Open
wants to merge 11 commits into
base: master
from

Conversation

Projects
None yet
2 participants
@frob
Copy link
Contributor

commented Dec 18, 2018

After reading #87 I realized that there seemed to be no non-filesystem based backup provisions. This MR adds them.

Adds the Makefile commands:

  • backup-lms
  • backup-cms
  • restore-lms
  • restore-cms

The restore commands allow users to specify the DUMP used for the restoration.

@frob

This comment has been minimized.

Copy link
Contributor Author

commented Dec 19, 2018

Documentation is coming.

@frob frob force-pushed the frob:backup-restore branch from 8486722 to b48a301 Dec 19, 2018

@frob frob force-pushed the frob:backup-restore branch from b48a301 to 5a10375 Dec 19, 2018

@frob

This comment has been minimized.

Copy link
Contributor Author

commented Dec 19, 2018

Okay, documentations is done. I have rebased off regisb/master I am doing some final testing before removing WIP.

@regisb

This comment has been minimized.

Copy link
Owner

commented Dec 19, 2018

This is a useful feature. I like how the backup and restore/restore-dangerous commands work together. My feeling is that the bash commands are a bit brittle, and too difficult to understand. Moreover, the backup commands only work for the openedx database, and not for the xqueue and the notes databases. But this PR is a good starting point. I'll have some time to work on this next week, during the Christmas holidays. In particular, I'd suggest renaming the make commands to something like backup-openedx-sql and backup-openedx-mongodb, since the lms and the cms essentially share the same database. Are you ok with that?

@frob

This comment has been minimized.

Copy link
Contributor Author

commented Dec 20, 2018

Good to know [about the other databases]. I was basing this off of the issue #87 so I thought those where the only two DBs that need the backup feature. I agree with you about the bash commands but I cannot think of a good way to do it in make and breaking it into shell scripts seemed a bit obfuscating.

The main difficulty is the mongp db backup. It would be easier if the mongodump utility was on the lms/cms images, but its not and I like to avoid adding dependencies for utilities. Another option to my approach is to just use the base ubuntu image and add everything we need in a single inline docker command; instead of relying on docker-compose.

@frob

This comment has been minimized.

Copy link
Contributor Author

commented Dec 20, 2018

I am looking through the services, I see these possibly needing backups:

  • lms
  • cms
  • xqueue
  • notes
  • forum

did I miss anything?

@regisb

This comment has been minimized.

Copy link
Owner

commented Dec 22, 2018

In fact, what we want to do is to backup MySQL data from lms/cms, notes, xqueue and the "openedx" database from Mongodb (see this answer on migrating from a native install that explains how to dump data from mysql and mongodb).

Maybe we can create backup and restore scripts, pretty much like the provision.sh script in configurator/templates/openedx/provision.sh? We could then run:

docker-compose run --rm -v ./data/backup:/openedx/backup/ lms /openedx/config/backup.sh /openedx/backup/$(date -u "+%Y%m%d-%H%M%S")

And for restoring:

docker-compose run --rm -v $(DUMP):/openedx/backup/ lms /openedx/config/restore.sh /openedx/backup/

What do you think?

@regisb

This comment has been minimized.

Copy link
Owner

commented Jan 30, 2019

@frob I don't know if you've resumed work on this, but I think it's better if you wait a little until the release of tutor v3. A lot of things are going to change, and you may have to throw away most of the things you did.

@frob

This comment has been minimized.

Copy link
Contributor Author

commented Jan 30, 2019

@regisb regisb force-pushed the regisb:master branch 5 times, most recently from 4cd12ce to 4331bc5 Feb 9, 2019

@regisb regisb force-pushed the regisb:master branch 3 times, most recently from e896b4b to f843bd6 Apr 22, 2019

@regisb regisb force-pushed the regisb:master branch 4 times, most recently from b528123 to 5558454 Apr 22, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.