Built in Database backup no longer works

[tylerryan]

The built in database backup no longer works since adodb has been replaced.
Implement a solution that uses mysqldump and fails gracefully if that system command is not able to be executed.

[aembler]

Yep. Exactly what Ryan said – I'd add that we should gray out the button and show a tooltip if mysqldump can't be found, etc... add a path to mysqldump in the configure.php and check it exists, etc...

This will actually be good, I think. I've held off on using the built-in backup stuff because I just didn't think it could handle a site of any size. Mysqldump is rock solid and I think having it use that as the engine is a good thing.

[joe-meyer]

I think mysqldump is a good stopgap solution, but I think that if we want to think a little further out we should be removing these sort of database engine dependencies if possible. It's not infrequent someone asks about using Concrete5 with their MSSQL database or postgres database and while that's not a target for c5 (yet), I think implementing a database dependent solution without more thought on it might end up wasting resources.

[appliculture]

https://github.com/heybigname/backup-manager maybe this could help ?

[aembler]

That looks really handy. That'd definitely be the way to go.

[TimDix]

Any update on this as far as ETA / Priority of this feature?

[joe-meyer]

The backup manager requires php >= 5.4 it looks like

[hissy]

Additionally, the backup manager requires mysqldump command.

[TimDix]

Sorry, not sure if we're talking about the same thing or not... I'm talking about /dashboard/system/backup/backup/run_backup

Which has this:

            //@TODO this backup stuff needs to be reworked since we're not using adodb anymore
            $this->set('message', t('This has not been implemented in 5.7'));
            return;

Which doesn't appear to be implemented at all.

https://github.com/concrete5/concrete5/blob/develop/web/concrete/controllers/single_page/dashboard/system/backup/backup.php#L16

[joe-meyer]

@TimDix same issue. If you read up there was a suggestion to make use of the backup-manager to do the backups and @aembler seemed to think this was the way to go.

[hissy]

How about this library? It's lightweight, no requirements, run without mysqldump command.

https://github.com/ifsnop/mysqldump-php

[aembler]

At a glance, it looks great

[joe-meyer]

@hissy it does look like a pretty decent library, my biggest concern are timeouts. The library from what little bit i've read doesn't really provide a good way for concrete5 to backup 1 table per request or similar to better prevent timeout situations.

I do think, however, that it might be an ideal candidate for a CLI command, with the hopes that those are less prone to timeout issues.

[hissy]

I think cli command to backup would be a good solution to timeout issue.

[joe-meyer]

Also.. .this may be scope creep on this issue.. But I think a backup level option would be a great improvement too... I'm thinking something like

1. Database Only
2. Database & Config
3. Database & Application Files
4. Database, Application & Package Files

Far to often someone attempts to backup their database, thinking that's all they need to migrate their site from one host to another. Also, allowing for a fuller approach to backups where one could restore to a specific point in time could have great advantages from a Disaster Recovery perspective.

[MrKarlDilkington]

@joe-meyer

These are excellent ideas.

Having true full backup and a timeline backup within the UI would be a real asset to users.

[hissy]

I've just started working of backup stuff.

https://github.com/hissy/concrete5-5.7/tree/enhancement/database-backup

I'd like to push it If we get feature/backup brunch into this repository. I think we need more discussion and more contributing from community.

[aembler]

I'd rather have on database backup that worked well and had a couple
options to exclude certain tables (logs, statistics, etc...) and simply
prompted the user to immediately download a zipped file when complete, then
a bunch of crazy options for backing up files, etc... I think a separate
package would work well for something like that.

On Tue, Aug 25, 2015 at 6:54 AM, Takuro Hishikawa notifications@github.com
wrote:

I've just started working of backup stuff.

https://github.com/hissy/concrete5-5.7/tree/enhancement/database-backup

I'd like to push it If we get feature/backup brunch into this repository.
I think we need more discussion and more contributing from community.

—
Reply to this email directly or view it on GitHub
#571 (comment)
.

[tduncandesign]

A package that does what Joe Meyer suggests would be great.
I've got a shell script I got from somewhere, for files + db, runs at X time each day, I just can't get it to clean up after itself -- ie, delete backups older than X days. Any chance you guys would like to see if you could clean that up? At least people could download it, make a cron job and have continuously working backups going.

[joe-meyer]

@tduncandesign I think the goal here is to build something that's deliverable from a concrete5 interface. A shell script kind of falls outside of that, especially considering that some people might run on different environments where a shell script isn't an option (think windows hosts or shared hosts that don't allow them). That said, I'm sure if you posted what you had to the concrete5 forums there would be others interested and possibly even willing to help you refine it, especially with the lack of a solid backup option right now.

[queesamor]

Is this something that the core team is going to make a decision on, or is it up to the community to make a decision, do the work, and then request approval from the core team? I ask because Andrew's comments seem supportive, but hands off...

[aembler]

Built-in backup ought to cover 70% of the cases, be easy to understand, and work well for sites of most size.

1. This has to work completely from a web interface.
2. This has to work without installing other libraries. If a common binary like mysqldump is required that might not be a dealbreaker, but it has to be checked for and the feature disabled if we can't find it.
3. This has to work without a lot of configuration.
4. I don't like the idea of .sql files in the file system. In 5.6 we encrypted them but I think it's just too much. I'd much rather hit a button, and immediately get a file to download. It'd also be cool if we could zip them up the way we do files in the file manager when you select bulk files to download.
5. I think having a couple options (exclude statistics, exclude logs) would be handy.
6. I don't want a full site backup. There are lots of options for that. If someone really wants that from concrete5, they can use an add-on. Or S3, or cPanel. This began as a Database Backup feature, let's just keep it that for now.

[chriswayg]

@aembler +1 to that

[githubsch]

Checkout the backup-options Drupal CMS offers: Adding notes to backups can be handy. Backups "in-the-air" /dropbox /NodeSquirrel (in Drupal) might be useful too. //backup entire site ..doesn't quite do it.

[KorvinSzanto]

Is backup manager still an option? If so, we should just go with that. We can enable backing up to external locations with config for power users and just expose the basics to end users with a button to download. It's 5.5.9+ so we'd need to do this in 8.0.0+

https://github.com/backup-manager/backup-manager

[hissy]

Backup Manager is a good solution, but it requires mysqldump and mysql command-line binaries.

[KorvinSzanto]

@hissy I'm not aware of a good solution that doesn't require command line executables (And I frankly wouldn't trust any).

[hissy]

Here is a one.
https://github.com/ifsnop/mysqldump-php

I agree that mysqldump is more reliable way, but we can fallback to no requirements (and less reliability) option. Because, I think users who really want this feature can not use mysqldump command on their shared hosting.

[aembler]

I'm not sure I believe that everyone who'd use this tool is only using it because mysqldump is specifically unavailable on their system. I imagine a large number of people a) don't know what mysqldump is b) don't know how to ssh into their server even if the server allows it c) just want to back up their database.

I originally liked the idea of this being mysqldump based because it's a rock solid utility and it's fast. We work with some large databases. If mysqldump-php is really as reliable then a pure PHP solution is definitely compelling – but I don't think one that is based off of mysqldump is bad. Far from it.

[KorvinSzanto]

@aembler agreed on all points. Lets do some testing and figure out where mysqldump won't work.

[gondwanasoft]

FWIW, I used mysqldump to do the dirty work in the 'Backup Database' job in this add-on. I'll probably get feedback about circumstances in which it won't work.

[krebbi]

give this a try: https://datatainment.de/this-n-that/concrete5-backup-and-restore-database