Mogbak makes it easy to backup your MogileFS domain to a single self contained directory. It has the ability to
update that directory again and again to match your MogileFS domain. This makes it possible for you to use
LVM snapshots. Mogbak can also fork worker processes to backup or restore files in parallel.
mogbak create --db=mogilefs --dbhost=mysqlserver --dbpass=secret --dbuser=mogile --domain=awesomeapp \\ --trackerip=10.10.10.10 --workers=10 /backups/awesomeapp mogbak backup /backups/awesomeapp
mogbak backup --workers=10 /backups/awesomeapp
mogbak list /backups/awesomeapp
mogbak restore --domain=restoreawesomeapp --trackerip=10.10.10.10 --workers=10 /backups/awesomeapp
mogbak restore --domain=restoreawesomeapp --trackerip=10.10.10.10 --single-file=abc1234file --workers=10 /backups/awesomeapp
MogileFS simply bumps its FID value in the files table when a new file is saved. This makes it quite simple for us to query and see what files need to be backed up since our last backup. The problem is that we also need to know what files have been deleted from MogileFS but still live within your backup. Since MogileFS has no delete log for us to look at we need to query the database in a brute-force manner. This would be extremely painful without access to the database. We do this as efficiently as we can, our cluster has about 3 million files and it takes less than a second. You can disable this feature with --no-delete switch.
The good news is that mogbak only needs SELECT access.
- db.sqlite - holds the metadata of each file in the backup
- settings.yml - holds the settings to connect to the mysql database and the tracker
- Backup files hashed using the same scheme as MogileFS Server
- Space. Obviously with large clusters the ability to save a full backup onto one device probably isn't possible
- Database. Right now only MySQL backed trackers are supported
There are certainly things that could be done about the above issues. Pull requests are welcome :)
- Ruby 1.9 is what we test against. It'll probably work under 1.8 but you'll be a guinea pig.
- mysql client development libraries (for mysql2 gem dependency)
- sqlite3 development libraries (for sqlite3 gem dependency)
gem install mogbak