Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Built in Database backup no longer works #571

Closed
tylerryan opened this issue Jul 31, 2014 · 31 comments

Comments

@tylerryan
Copy link
Contributor

@tylerryan tylerryan commented Jul 31, 2014

The built in database backup no longer works since adodb has been replaced.
Implement a solution that uses mysqldump and fails gracefully if that system command is not able to be executed.

@aembler

This comment has been minimized.

Copy link
Member

@aembler aembler commented Jul 31, 2014

Yep. Exactly what Ryan said – I'd add that we should gray out the button and show a tooltip if mysqldump can't be found, etc... add a path to mysqldump in the configure.php and check it exists, etc...

This will actually be good, I think. I've held off on using the built-in backup stuff because I just didn't think it could handle a site of any size. Mysqldump is rock solid and I think having it use that as the engine is a good thing.

@aembler aembler added this to the 5.7.1 milestone Jul 31, 2014
@aembler aembler modified the milestones: 5.7.1, 5.7.2 Oct 6, 2014
@joe-meyer

This comment has been minimized.

Copy link
Contributor

@joe-meyer joe-meyer commented Feb 19, 2015

I think mysqldump is a good stopgap solution, but I think that if we want to think a little further out we should be removing these sort of database engine dependencies if possible. It's not infrequent someone asks about using Concrete5 with their MSSQL database or postgres database and while that's not a target for c5 (yet), I think implementing a database dependent solution without more thought on it might end up wasting resources.

@aembler aembler modified the milestones: Feature Release, Future Mar 30, 2015
@appliculture

This comment has been minimized.

Copy link
Contributor

@appliculture appliculture commented Apr 15, 2015

@aembler

This comment has been minimized.

Copy link
Member

@aembler aembler commented Apr 16, 2015

That looks really handy. That'd definitely be the way to go.

@TimDix

This comment has been minimized.

Copy link
Contributor

@TimDix TimDix commented Jul 25, 2015

Any update on this as far as ETA / Priority of this feature?

@joe-meyer

This comment has been minimized.

Copy link
Contributor

@joe-meyer joe-meyer commented Jul 25, 2015

The backup manager requires php >= 5.4 it looks like

@hissy

This comment has been minimized.

Copy link
Contributor

@hissy hissy commented Jul 26, 2015

Additionally, the backup manager requires mysqldump command.

@TimDix

This comment has been minimized.

Copy link
Contributor

@TimDix TimDix commented Jul 26, 2015

Sorry, not sure if we're talking about the same thing or not... I'm talking about /dashboard/system/backup/backup/run_backup

Which has this:

            //@TODO this backup stuff needs to be reworked since we're not using adodb anymore
            $this->set('message', t('This has not been implemented in 5.7'));
            return;

Which doesn't appear to be implemented at all.

https://github.com/concrete5/concrete5/blob/develop/web/concrete/controllers/single_page/dashboard/system/backup/backup.php#L16

@joe-meyer

This comment has been minimized.

Copy link
Contributor

@joe-meyer joe-meyer commented Jul 26, 2015

@TimDix same issue. If you read up there was a suggestion to make use of the backup-manager to do the backups and @aembler seemed to think this was the way to go.

@hissy

This comment has been minimized.

Copy link
Contributor

@hissy hissy commented Aug 21, 2015

How about this library? It's lightweight, no requirements, run without mysqldump command.

https://github.com/ifsnop/mysqldump-php

@aembler

This comment has been minimized.

Copy link
Member

@aembler aembler commented Aug 21, 2015

At a glance, it looks great

@joe-meyer

This comment has been minimized.

Copy link
Contributor

@joe-meyer joe-meyer commented Aug 25, 2015

@hissy it does look like a pretty decent library, my biggest concern are timeouts. The library from what little bit i've read doesn't really provide a good way for concrete5 to backup 1 table per request or similar to better prevent timeout situations.

I do think, however, that it might be an ideal candidate for a CLI command, with the hopes that those are less prone to timeout issues.

@hissy

This comment has been minimized.

Copy link
Contributor

@hissy hissy commented Aug 25, 2015

I think cli command to backup would be a good solution to timeout issue.

@joe-meyer

This comment has been minimized.

Copy link
Contributor

@joe-meyer joe-meyer commented Aug 25, 2015

Also.. .this may be scope creep on this issue.. But I think a backup level option would be a great improvement too... I'm thinking something like

  1. Database Only
  2. Database & Config
  3. Database & Application Files
  4. Database, Application & Package Files

Far to often someone attempts to backup their database, thinking that's all they need to migrate their site from one host to another. Also, allowing for a fuller approach to backups where one could restore to a specific point in time could have great advantages from a Disaster Recovery perspective.

@MrKarlDilkington

This comment has been minimized.

Copy link
Contributor

@MrKarlDilkington MrKarlDilkington commented Aug 25, 2015

@joe-meyer

These are excellent ideas.

Having true full backup and a timeline backup within the UI would be a real asset to users.

@hissy

This comment has been minimized.

Copy link
Contributor

@hissy hissy commented Aug 25, 2015

I've just started working of backup stuff.

https://github.com/hissy/concrete5-5.7/tree/enhancement/database-backup

I'd like to push it If we get feature/backup brunch into this repository. I think we need more discussion and more contributing from community.

@aembler

This comment has been minimized.

Copy link
Member

@aembler aembler commented Aug 25, 2015

I'd rather have on database backup that worked well and had a couple
options to exclude certain tables (logs, statistics, etc...) and simply
prompted the user to immediately download a zipped file when complete, then
a bunch of crazy options for backing up files, etc... I think a separate
package would work well for something like that.

On Tue, Aug 25, 2015 at 6:54 AM, Takuro Hishikawa notifications@github.com
wrote:

I've just started working of backup stuff.

https://github.com/hissy/concrete5-5.7/tree/enhancement/database-backup

I'd like to push it If we get feature/backup brunch into this repository.
I think we need more discussion and more contributing from community.


Reply to this email directly or view it on GitHub
#571 (comment)
.

@tduncandesign

This comment has been minimized.

Copy link

@tduncandesign tduncandesign commented Aug 25, 2015

A package that does what Joe Meyer suggests would be great.
I've got a shell script I got from somewhere, for files + db, runs at X time each day, I just can't get it to clean up after itself -- ie, delete backups older than X days. Any chance you guys would like to see if you could clean that up? At least people could download it, make a cron job and have continuously working backups going.

@joe-meyer

This comment has been minimized.

Copy link
Contributor

@joe-meyer joe-meyer commented Aug 25, 2015

@tduncandesign I think the goal here is to build something that's deliverable from a concrete5 interface. A shell script kind of falls outside of that, especially considering that some people might run on different environments where a shell script isn't an option (think windows hosts or shared hosts that don't allow them). That said, I'm sure if you posted what you had to the concrete5 forums there would be others interested and possibly even willing to help you refine it, especially with the lack of a solid backup option right now.

@queesamor

This comment has been minimized.

Copy link

@queesamor queesamor commented Sep 6, 2015

Is this something that the core team is going to make a decision on, or is it up to the community to make a decision, do the work, and then request approval from the core team? I ask because Andrew's comments seem supportive, but hands off...

@aembler

This comment has been minimized.

Copy link
Member

@aembler aembler commented Sep 11, 2015

Built-in backup ought to cover 70% of the cases, be easy to understand, and work well for sites of most size.

  1. This has to work completely from a web interface.
  2. This has to work without installing other libraries. If a common binary like mysqldump is required that might not be a dealbreaker, but it has to be checked for and the feature disabled if we can't find it.
  3. This has to work without a lot of configuration.
  4. I don't like the idea of .sql files in the file system. In 5.6 we encrypted them but I think it's just too much. I'd much rather hit a button, and immediately get a file to download. It'd also be cool if we could zip them up the way we do files in the file manager when you select bulk files to download.
  5. I think having a couple options (exclude statistics, exclude logs) would be handy.
  6. I don't want a full site backup. There are lots of options for that. If someone really wants that from concrete5, they can use an add-on. Or S3, or cPanel. This began as a Database Backup feature, let's just keep it that for now.
@chriswayg

This comment has been minimized.

Copy link

@chriswayg chriswayg commented Sep 20, 2015

@aembler +1 to that

@githubsch

This comment has been minimized.

Copy link

@githubsch githubsch commented Dec 3, 2015

Checkout the backup-options Drupal CMS offers: Adding notes to backups can be handy. Backups "in-the-air" /dropbox /NodeSquirrel (in Drupal) might be useful too. //backup entire site ..doesn't quite do it.
bildschirmfoto 2015-12-03 um 23 02 22

@KorvinSzanto

This comment has been minimized.

Copy link
Member

@KorvinSzanto KorvinSzanto commented Jan 10, 2016

Is backup manager still an option? If so, we should just go with that. We can enable backing up to external locations with config for power users and just expose the basics to end users with a button to download. It's 5.5.9+ so we'd need to do this in 8.0.0+

https://github.com/backup-manager/backup-manager

@hissy

This comment has been minimized.

Copy link
Contributor

@hissy hissy commented Jan 11, 2016

Backup Manager is a good solution, but it requires mysqldump and mysql command-line binaries.

@KorvinSzanto

This comment has been minimized.

Copy link
Member

@KorvinSzanto KorvinSzanto commented Jan 11, 2016

@hissy I'm not aware of a good solution that doesn't require command line executables (And I frankly wouldn't trust any).

@hissy

This comment has been minimized.

Copy link
Contributor

@hissy hissy commented Jan 11, 2016

Here is a one.
https://github.com/ifsnop/mysqldump-php

I agree that mysqldump is more reliable way, but we can fallback to no requirements (and less reliability) option. Because, I think users who really want this feature can not use mysqldump command on their shared hosting.

@aembler

This comment has been minimized.

Copy link
Member

@aembler aembler commented Jan 11, 2016

I'm not sure I believe that everyone who'd use this tool is only using it because mysqldump is specifically unavailable on their system. I imagine a large number of people a) don't know what mysqldump is b) don't know how to ssh into their server even if the server allows it c) just want to back up their database.

I originally liked the idea of this being mysqldump based because it's a rock solid utility and it's fast. We work with some large databases. If mysqldump-php is really as reliable then a pure PHP solution is definitely compelling – but I don't think one that is based off of mysqldump is bad. Far from it.

@KorvinSzanto

This comment has been minimized.

Copy link
Member

@KorvinSzanto KorvinSzanto commented Jan 11, 2016

@aembler agreed on all points. Lets do some testing and figure out where mysqldump won't work.

@gondwanasoft

This comment has been minimized.

Copy link

@gondwanasoft gondwanasoft commented Sep 21, 2017

FWIW, I used mysqldump to do the dirty work in the 'Backup Database' job in this add-on. I'll probably get feedback about circumstances in which it won't work.

@krebbi

This comment has been minimized.

Copy link
Contributor

@krebbi krebbi commented Sep 21, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
You can’t perform that action at this time.