Permalink
Browse files

Initial Import

  • Loading branch information...
1 parent 3f49782 commit 370ff3cad64b65e8029488e7de32a10891611aec derks committed Apr 13, 2010
Showing 502 changed files with 70,329 additions and 1 deletion.
View
@@ -0,0 +1,3 @@
+*pyc
+*egg*
+build/
View
235 ChangeLog
@@ -0,0 +1,235 @@
+2009-12-04 BJ Dierkes <wdierkes@rackspace.com>
+ * Added ./devtools/build_rpms.py to auto build rpms
+ from trunk.
+
+2009-12-01 Andrew Garner <andrew.garner@rackspace.com>
+ * Improved logging when plugins fail unexpectedly
+
+2009-11-24 Andrew Garner <andrew.garner@rackspace.com>
+ * holland backup --dry-run now checks available disk
+ space for whatever device /var/spool/hollland is on
+ compared to whatever the backup plugin reports
+ in plugin.estimate_backup_size()
+
+2009-11-13 Andrew Garner <andrew.garner@rackspace.com>
+ * holland log format is no longer hardcoded
+ * Use --base-dir rather than --basedir on new versions
+ of maatkit [#1554]
+
+2009-11-12 Andrew Garner <andrew.garner@rackspace.com>
+ * More cleanups for mysqldump restore plugin
+ * Add RHEL4 fixes for LVM plugin
+
+2009-11-08 Andrew Garner <andrew.garner@rackspace.com>
+ * LVM Backup plugin added [#1372]
+ * holland restore command added
+ * implemented mysqldump restore plugin
+ * implemented lvm restore plugin
+
+2009-11-08 Jason Kölker <jkoelker@rackspace.com>
+ * holland purge now asks for permission unless
+ the --force flag is used [#1541]
+ * Added Google logo to tomsay [#1543]
+
+2009-11-07 Andrew Garner <andrew.garner@rackspace.com>
+ * The mysqldump backup plugin now logs the database
+ and table filter patterns before starting backups
+ [#1539]
+ * holland list-plugins now lists holland.backup and
+ holland.commands plugins rather than just the
+ distributions.
+ * Raised the idle timeout in the mysqldump plugin from
+ 8 hours to 24 hours [#1537].
+
+2009-10-26 Andrew Garner <andrew.garner@rackspace.com>
+ * Cleaned up output and several bugfixes in mk-config
+ command [#1510]
+
+2009-10-26 Andrew Garner <andrew.garner@rackspace.com>
+ * Fixed table include/exclude patterns to support
+ both table and database.table matching [#1502]
+
+2009-10-09 Andrew Garner <andrew.garner@rackspace.com>
+ * Quoted values in my.cnf files are now parsed properly
+ [#1473]
+
+2009-09-15 Jason Kölker <jkoelker@rackspace.com>
+ * Improved commvault argument handling [#1466,#1471]
+ * Allow commvault to provide explicit backupsets
+
+2009-09-14 BJ Dierkes <wdierkes@rackspace.com>
+ * Holland 0.9.4 released.
+
+2009-09-11 Andrew Garner <andrew.garner@rackspace.com>
+ * Updated maatkit backup plugin to support option name
+ changes in mk-parallel-dump. Resolves [#1390]
+
+2009-08-26 Andrew Garner <andrew.garner@rackspace.com>
+ * Improved error message for config file syntax errors.
+ Resolves [#1443]
+ * Now Holland's backup module always logs the final
+ size estimated after applying the estimated-size-factor
+ adjustment. Resolves [#1399]
+ * In the mysqldump backup plugin, if bin-log-position is
+ requested on a slave, but log-bin is not enabled a warning
+ is logged, but this is no longer treated as an error.
+ Resolves [#1394]
+
+2009-08-20 Andrew Garner <andrew.garner@rackspace.com>
+ * Improved error message when STOP SLAVE fails, but
+ stop-slave was requested in the mysqldump backup
+ plugin. Resolves [#1435]
+
+2009-07-28 Andrew Garner <andrew.garner@rackspace.com>
+ * Improved commvault option parsing.
+
+2009-07-24 Andrew Garner <andrew.garner@rackspace.com>
+ * stderr output from compression pipes is now captured
+ and logged. Resolves [#1395]
+
+2009-07-23 Andrew Garner <andrew.garner@rackspace.com>
+ * When running the mysqldump backup plugin in dry-run
+ mode the slave is no longer stopped. Resolves [#1396]
+ * MySQL information_schema based database/table filtering
+ will filter databases more efficiently now.
+ Resolves [#1397]
+ * Using binlog-pos on a slave no longer requires binary
+ logging to be enabled.
+
+2009-07-13 BJ Dierkes <wdierkes@rackspace.com>
+ * Holland 0.9.3 released.
+
+
+2009-07-08 Andrew Garner <andrew.garner@rackspace.com>
+ * Fixed a couple bugs in holland.commands.mysql where
+ mysql-extract would fail. Resolves [#1379]
+ * Made estimated backup size adjustments more
+ consistent. [holland:backup] now accepts
+ a 'estimated-size-factor' that is a decimal
+ multiplier (default 1.0). This will apply
+ to any provider without any special support.
+ This also removes free-space-factor from the
+ mysqldump and mysqlhotcopy plugins and their
+ example configs. Resolves [#1378]
+ * Added 'backups-to-keep=1' to holland mk-config
+ generated configs. Resolves [#1374]
+ * Strip unknown opts when generating a my.cnf with
+ make_mycnf in holland.lib.mysql. Resolves [#1377]
+ * Fix defaults-extra-file typo in Maatkit plugin.
+ Resolves [#1377]
+ * Fix holland_cvmysqlsv to return correct exit code.
+ Resolves [#1375]
+ * Fix dry-run when using file-per-database option
+ Resolves [#1373]
+
+2009-06-11 BJ Dierkes <wdierkes@rackspace.com>
+ * Cleaned up --help output a bit. Resolves [#1333]
+ * Holland 0.9.2 released.
+
+2009-06-03 BJ Dierkes <wdierkes@rackspace.com>
+ * Holland 0.9.1 released.
+
+2009-05-06 Andrew Garner <andrew.garner@rackspace.com>
+ * Added holland.lib.mysql.client.connect method. This
+ rewrites my.cnf parameters to what MySQLdb.connect
+ expects. This allows passing option files from
+ holland.lib.mysql.option to the client.connect method
+ and connecting without necessarily flushing the options
+ to disk and using read_default_file=<path>.
+
+2009-04-19 Andrew Garner <andrew.garner@rackspace.com>
+ * Holland 0.3 released.
+
+2009-04-19 Andrew Garner <andrew.garner@rackspace.com>
+ * Converted holland.lib.mysql.client testcases to use
+ mock objects instead of direct dbapi connections
+ * Added Ian Bicking's multidict implementation to
+ holland.lib.common (used by holland.lib.mysql.option
+ but probably generally useful)
+ * mk-config now adds a [holland:backup] section to
+ generated configs
+
+2009-04-17 Andrew Garner <andrew.garner@rackspace.com>
+ * The mysqldump plugin now logs SHOW SLAVE STATUS
+ replication information to [mysql:replication] in
+ the backup.conf rather than to a separate replication.conf
+ file.
+ * Added "holland purge" command
+ * The holland.lib.mysql plugin distribution no longer lists
+ a setuptools dependency on MySQL-Python. This was breaking
+ on OS X in some cases and overall slowing down test runs.
+ * Fixed a compatibility problem in validate.py that was preventing
+ holland from running on python-2.3 (RHEL4)
+ * Further improvements to [.]my.cnf option-file processing in
+ holland.lib.mysql
+ * mysqlhotcopy and maatkit plugins now use the new my.cnf interface
+ in holland.lib.mysql.option
+ * /etc/holland/holland.conf is the default config file location
+ if an explicit --config is not specified or HOLLAND_CONFIG is
+ not set.
+ * [COMPATIBILITY] backupsets are now required to specify a
+ [holland:backup] section with a 'plugin' parameter to specify
+ the plugin to use (mysqldump,mysqlhotcopy,maatkit,...) rather
+ than a "root-level" provider option. This allows holland's config
+ files to be processed by non-ConfigObj standard ini-tools.
+ Further, [holland:backup] provides a bucket to record other
+ backup information about the backupset in the saved backup.conf
+ in the spool directory.
+ * Fixed a problem in "holland mk-config" that didn't generate optional
+ default=None parameters correctly.
+ * Fixed a minor rendering bug in Commmand.py's help formatting.
+ * Fixed a bug in holland help when referencing a command that does not
+ exist.
+ * Lots of bugfixes
+ * Coalesced holland.lib.archive and holland.lib.compression into
+ holland.lib.common distribution, along with a few other modules
+ that were previously in holland-core (but unused by holland-core)
+
+2009-04-16 Andrew Garner <andrew.garner@rackspace.com>
+ * Upgraded to ConfigObj 4.6.0, validate.py 1.0.0
+ * Cleaned up holland-core structure
+ * Added several in-progress plugins
+ * Moved many of the development "helper" scripts in trunk/
+ to trunk/dev-util/
+
+2009-04-01 Andrew Garner <andrew.garner@rackspace.com>
+ * Fixed maatkit plugin logging to capture stderr output
+ when running mk-parallel-dump
+
+2009-03-27 Andrew Garner <andrew.garner@rackspace.com>
+ * Added "holland mk-config" command to generate a backupset
+ config file.
+ * The mysqldump plugin now saves replication information from
+ SHOW SLAVE STATUS to replication.conf when backing up a slave
+ if bin-log-position is also defined.
+
+2009-03-24 Andrew Garner <andrew.garner@rackspace.com>
+ * Updated [.]my.cnf option-file processing in holland.lib.mysql
+ * Improved the performance of mysqlhotcopy backup (Issue #1191)
+ * Made stopping/starting slaves with mysqldump plugin more robust
+ * Added "defaults-extra-file" parameter to mysqldump plugin
+ This takes a list of my.cnf files which are used with the
+ backupset's mysqlclient config to produce a my.cnf auth-file.
+
+2009-03-11 Andrew Garner <andrew.garner@rackspace.com>
+ * Added "holland list-backups" command to show available backups
+ * Fixed backup purging functionality (Issue #1190)
+
+2009-03-10 Andrew Garner <andrew.garner@rackspace.com>
+
+ * Holland 0.2 release
+ * Bugfixes for Holland-Core
+ * Renamed holland.providers to holland.backup
+ * Refactored MySQLDump Backup Plugin; Now supports
+ advanced database and table filtering and many
+ new features previously broken in the config file
+ are now enabled
+ * Better logging throughout framework and plugins
+ * Enable dry-run support; Supported in all backup plugins
+ * Added MySQLHotcopy backup plugin
+ * Added Archive library plugin (used by MySQLHotcopy)
+ * Added Maatkit backup plugin (uses mk-parallel-dump)
+
+2009-02-20 Andrew Garner <andrew.garner@rackspace.com>
+
+ * Holland 0.1 released
View
57 INSTALL
@@ -0,0 +1,57 @@
+FIXME: This is rough and to the point.... needs fluff.
+
+
+UNPACK THE SOURCE:
+
+ you@linuxbox ~]$ tar -zxvf holland-x.y.z.tar.gz
+
+
+LOCAL DEVELOPMENT INSTALLATION:
+
+ you@linuxbox ~] cd ./holland-x.y.z
+ you@linuxbox holland-x.y.z] ./devtools/mkvirtenv.sh
+
+ Optionally, for target testing... add the --no-deps option to force
+ holland to rely on system dependencies (such as MySQL-python).
+
+
+SOURCE INSTALLATION:
+
+ Install Holland Core:
+
+ you@linuxbox ~]$ cd ./holland-x.y.z/holland-core
+ you@linuxbox holland-core]$ sudo python setup.py install
+
+
+ Install Holland Core Plugins:
+
+ you@linuxbox holland-x.y.z]$ cd plugins/holland.lib.common
+ you@linuxbox holland.lib.common]$ sudo python setup.py install
+
+ you@linuxbox holland-x.y.z]$ cd plugins/holland.lib.mysql
+ you@linuxbox holland.lib.mysql]$ sudo python setup.py install
+
+
+ Install Optional Plugins (see plugins/ACTIVE):
+
+ you@linuxbox holland-x.y.z]$ cd plugins/<plugin_dir>
+ you@linuxbox holland.<plugin_dir>]$ sudo python setup.py install
+
+
+
+UBUNTU INSTALLATION:
+
+ you@linuxbox holland-x.y.z]$ cp -a distribution/ubuntu/ubuntu.jaunty debian
+ you@linuxbox holland-x.y.z]$ dpkg-buildpackage -us -uc
+
+
+
+REDHAT/FEDORA/CENTOS/ETC INSTALLATION:
+
+ root@linuxbox ~]# cp -a holland-x.y.z/distribution/holland.spec /usr/src/redhat/SPECS
+ root@linuxbox ~]# cp -a holland-x.y.z.tar.gz /usr/src/redhat/SOURCES
+ root@linuxbox ~]# cd /usr/src/redhat
+ root@linuxbox /usr/src/redhat]# rpmbuild -ba SPECS/holland.spec
+
+
+
View
1 README
@@ -1 +0,0 @@
-Placeholder
View
No changes.
View
@@ -0,0 +1,72 @@
+Here are three approaches to a config file format:
+- Hierarchical
+Sections are organized in a single file in a hierarchy
+A backup-set can have sub-sections that define the config
+for various plugins:
+ ...
+ [backupsets]
+ [[default_backup]]
+ key = value
+ ...
+ [[[mysql-plugin]]]
+ host = remote-host
+ ...
+
+- Directory
+Hierarchy is broken down into different files in
+different directories:
+holland.conf
+# backupset configs
+backupsets/
+ default_backup.conf
+# global backup-plugin configs
+providers/
+ mysqldump.conf
+ lvm.conf
+ pgdump.conf
+ ...
+# global helper plugin library configs
+helpers/
+ mysqlclient.conf
+ lvmlib.conf
+ ...
+
+- Namespaces
+Single file, with standard ini sections named in a special
+way to denote namespaces.
+holland.conf:
+[holland]
+backup_directory = /<somepath>/
+
+[holland:logging]
+filename = /var/log/holland.log
+...
+
+[backup:default_backup]
+provider = mysqldump
+<mysqldump settings for 'default_backup'>
+[backup:default_backup:mysqlclient]
+# mysqldump backups utilize the mysqlclient lib
+host = 192.168.100.15 # some remote host
+user = remote-backup-user
+password = ...
+[backup:default_backup:compression]
+# suppose some general compression plugin we want to override
+cmd = lzop -U --fast
+
+[plugins:backup:mysqldump]
+# global options for the mysqldump backup plugin ("provider")
+# these are overriden in [backup:<name>]
+file_per_database = yes
+# exclude_databases = test
+# include_databases = production_db
+# regex = production_*
+# extra_args = --flush-privileges --ignore-table production_db.sessions
+
+[plugins:lib:mysqlclient]
+host = localhost
+socket = /var/lib/mysql/mysql.sock
+user = backup_user
+password = password
+ssl = no
+compress = no
Oops, something went wrong.

0 comments on commit 370ff3c

Please sign in to comment.