• Feb 20, 2015

    v1.4.1

    ===================
    v1.4.1 Release Notes
    ====================
    
    support for more secure Hub IAM role type temporary credentials
    ---------------------------------------------------------------
    
    * implemented internal stsagent command
    
    * added support for multiple credential types (e.g., devpay, iamrole,
      iamuser)
    
    * updated dependencies: newer version of tklbam-python-boto with STS
      agent support, python-dateutil (using this to parse
    
    * fallback to local registry path if global paths in non-writeable paths
    
      rational:
    
      1) allow tklbam to be used by a non-root user
      2) make testing easier
    
    * duplicity.py: allow graceful degradation of functionality when not running as root
    
    * backup: fatal error on bad subapikey if no cached credentials or iamrole credentials
    
    * added python-dateutil dependency
    
    improve generic Debian/Ubuntu support
    -------------------------------------
    
    * ez-apt-install.sh: cut and paste convenience script that can be piped from wget
    
      Rational: make installation on Ubuntu/Debian ridiculously easy
    
      Usage::
    
          # I feel lucky
          wget -O - -q https://raw.github.com/turnkeylinux/tklbam/master/contrib/ez-apt-install.sh | PACKAGE=tklbam /bin/bash
    
          # I feel cautious
          wget https://raw.github.com/turnkeylinux/tklbam/master/contrib/ez-apt-install.sh
          cat ez-apt-install.sh
          chmod +x
          ./ez-apt-install.sh tklbam
    
    * created detect-profile internal command
    
      rational: primitive for generic Ubuntu/Debian support
    
    * profile_id detection for non-TurnKey systems
    
    documentation improvements
    --------------------------
    
    * fixed man page for tklbam-init.txt to be in sync with embedded usage
    
    * added Features section to docs/tklbam.txt
    
    * tweaked and improved FAQ, added question on how to tune a backup
    
    * updated FAQ to clarify supported Linux systems, storage backends and databases
    
    * committed fix to fixclock hook to explain how to change ntp server
    
    debugging improvements
    ----------------------
    
    * removed some of the obstacles to running tklbam as a non-root user
      (e.g., useful for testing/debugging, or with --raw-upload /
      --raw-download) mode
  • Nov 11, 2013

    v1.4

    ==================
    v1.4 Release Notes
    ==================
    
    This release focused mainly on improving TKLBAM's usability, making it
    more robust, and improving it as an independent general purpose tool.
    
    Motivation: more people would find TKLBAM useful if...
    ------------------------------------------------------
    
    * If it worked on other Linux distributions (e.g., Debian and Ubuntu to
      begin with)
    
    * If users understood how it worked and realized they were in control.
      Magic is scary in a backup tool.
    
    * If it worked without the TurnKey Hub or better yet without needing a
      network connection at all.
    
    * If users realized that TKLBAM works with all the usual non-cloud storage
      back-ends such as the local filesystem, rsync, ftp, ssh, etc.
    
    * If users could more easily tell when something is wrong, diagnose the
      problem and fix it without having to go through TKLBAM's code or
      internals
    
    * If users could mix and match different parts of TKLBAM as required (e.g.,
      the part that identifies system changes, the part that interfaces with
      Duplicity to incrementally update their encrypted backup archives,
      etc.)
    
    * If users could embed TKLBAM in their existing backup solutions
    
    * If users realized TKLBAM allowed them to backup different things at
      different frequencies (e.g., the database every hour, the code every
      day, the system every week)
    
      Monolithic all-or-nothing system-level backups are not the only way to
      go.
    
    * If it could help with broken migrations (e.g., restoring a backup from
      TurnKey Redmine 12 to TurnKey Redmine 13)
    
    * If it worked more robustly, tolerated failures, and with fewer bugs
    
    Changes
    -------
    
    * TKLBAM no longer needs the TurnKey Hub or even a network connection
    
      - tklbam-init --solo option
    
      - tklbam-backup and tklbam-restore can now work without being linked
        to the TurnKey Hub
    
    * More powerful --force-profile option in init, backup and restore
    
      - The --force-profile=empty option allows you to force an empty
        profile, which is useful if:
    
        * you don't want TKLBAM to do a full system backup (e.g., you only
          want to backup a single directory)
    
        * TKLBAM can't download a profile to auto-configure itself (e.g.,
          unsupported appliance or Linux distribution)
    
        * you want full control over the configuration of the backup
    
        * don't want to link TKLBAM to a Hub account (tklbam-init --solo)
    
      - The --force-profile option accepts appliance codenames (e.g.,
        --force-profile=core). This makes --force-profile much easier to
        use. Previously the only option was to give it the exact profile
        version (e.g., ---force-profile=turnkey-core-13.0-wheezy-amd64)
    
      - The --force-profile option now accepts custom profiles such as
        those created by the "tklbam-internal create-profile" command.
    
      - tklbam-init and tklbam-restore now support the --force-profile
        option for setting a non-default backup profile. Previously, only
        tklbam-backup accepted the --force-profile option.
    
    * Increase the robustness of MySQL backup/restore by detecting and
      working around broken debian.cnf files, which can sometimes get out of
      sync with the mysql user database.
    
    * Improved logging
    
      - Log backup/restore output in real-time to /var/log
    
        Previously the logfile would only be written when backup/restore
        finished, which made it hard to diagnose problems with
        non-interactive runs of tklbam-backup (e.g.,
        /etc/cron.daily/tklbam-backup)
    
      - Log exception tracebacks
    
      - Added a log rotation script
    
    * Usability improvements make TKLBAM more verbose and self-documenting
    
      Goals:
    
      - make TKLBAM easier to understand, less magical
    
      - make it easier to tell when something is wrong, diagnose the
        problem and fix it without having to go through TKLBAM's code or
        internals
    
      - explain that TKLBAM works with all the usual non-cloud storage
        back-ends such as the local filesystem, rsync, ftp, ssh, etc. many
        users probably didn't realize these worked
    
      - better explain the things users don't realize are possible
            don't expect users to read every bit of documentation first
    
      Highlights:
    
      - tklbam-backup now explains what files and packages have changed,
        what databases will be backed up, etc.
    
      - illustrative usage examples
    
      - run help output through pager
    
      - list URLs for custom --address storage backends we know work
    
      - replace cryptic error messages with more helpful suggestions
    
      - embedded usage help goes through pager (e.g., less)
    
    * better --debug behavior
    
      - print an explanation before dropping you into the debug shell
    
      - fix a PATH override issue which prevented duplicity from working in
        the debug shell
    
      - added another "breakpoint" when using restore --debug, right before
        duplicity is executed
    
    * Cryptographically signed profile-level hooks are now supported. It's a
      generic mechanism but the original motivation for this was to allow us
      to automatically fix migrations between two versions of an appliance
      (E.g., TurnKey Redmine 12 and TurnKey Redmine 13)
    
    * At least 17 bugfixes including...
    
      - restore --simulate would overwrite debian.cnf, breaking mysql
      - chmod 700 /TKLBAM
  • Sep 9, 2013

    v1.3

    ==================
    v1.3 Release Notes
    ==================
    
    * Database support
    
      - PostgreSQL support (tested on Squeeze and Wheezy)
      - MySQL support for views and triggers
    
    * Improved backup
    
      - Added --raw-upload option
      - Added --dump option (dumps backup extract to directory)
      - Removed -s shorthand for --simulate
    
    * Improved restore
    
      - Added --raw-download option
      - Added --simulate option
      - Reduced noise in output
      - Support restoring backup extract
    
    * Hooks
    
      - Added "inspect" state to hooks mechanism
      - Added tklbam-hooks documentation
    
    * Updated and expanded (e.g., FAQ)
    
    * Bugfixes
    
      - backup
    
        - ignore Unix sockets
        - didn't catch Hub API exceptions (e.g., unresolvable hostname)
        - fixed fallback to cached values when Hub is unreachable
        - register socket into fsdelta as a status path, not an overwrite
        - chunkify repeats a chunk if the last element is larger than maxlen
        - backup bugfix: --quiet shouldn't cancel --simulate
        - backup pre hook should run before backup starts
    
      - restore
    
        - fix permissions and ownership for empty directories
        - apply overlay non-destructively by copying rather than moving files
        - fixed usage description for --time
        - mysql doesn't like it when you drop the mysql database
        - don't move files to rollback if they don't exist in the overlay
        - fixed tklbam-restore usage regression: --l => --limits
    
    * Development
    
      - Fixed regtest and made more robust, added README
      - Major refactorings (code readability and scalability)
      - removed HTML and manpages from docs/ (these are automatically generated files)
  • Oct 10, 2012

    1.1

    1.1 release on old master branch
    
  • Aug 10, 2012

    v1.2

    ==================
    v1.2 Release Notes
    ==================
    
    * Restore
    
      - added embedded squid download cache to make it easier to retry
        failed multi-GB restores without resorting to suicide
    
      - fixed root cause of the mysql max packet issue by limiting extended
        insert to 1MB chunks
    
    * Backup
    
      - session resume allows you to recover from aborted backup sessions
      - multipart parallel S3 uploads
      - skip backup components
      - finer full-backup frequencies (hourly, minutely)
      - don't upload asynchronously (slower but less confusing)
      - added --force-profile option to make it easier to force a different
        backup profile
    
    * General
    
        TKLBAM_CONF and TKLBAM_REGISTRY environment variables allow multiple
        configurations of TKLBAM to be run on the same machine
      - embedded new versions of duplicity (0.6.18) and python-boto (2.3.0)
    
    * Development
    
      - added --debug option for backup and restore commands
      - added wrappers to internal commands
    
    * Bugfixes:
    
      - log TKLBAM exceptions
      - many others
  • Aug 8, 2012

    v1.0

    ==================
    v1.0 Release Notes
    ==================
    
    * Fully implemented. Stick a fork in me I'm done!
  • Aug 8, 2012

    v1.1

    ==================
    v1.1 Release Notes
    ==================
    
    * UI fixes
    
      - init
    
        - aded --force option to support re-initialization of TKLBAM
        - increased robustness of stdin input of APIKEY
        - added API-KEY validity check to tklbam-init (save round-trip)
        - catch scary looking tracebacks on initialization
    
      - backup
    
        - handle BackupAccount.NotFound error, raise NotSubscribedError
        - removed --asynchronous-upload option (trade off better performance
          on high speed networks with improved transparency)
        - If user isn't subscribed, print out instructions, not a traceback
    
      - restore:
    
        - usepty=True in restore to prevent dpkg output from going crazy
        - support --noninteractive mode (e.g., don't prompt or retry passphrases)
    
      - added new status command (improve TKLBAM-Core integration)
      - escrow: added -R flag as shortcut for --random-passphrase (convenience)
      - list: size in megabytes instead of gigabytes (better resolution)
    
    * bugfixes
    
      - increase nofile rlimit to 8192 (needed by long backup chains)
      - don't tell the Hub to update the backup in simulate mode!
      - add --allow-source-mismatch option to duplicity (otherwise Duplicity
        refuses to backup if the hostname changes)
      - if we skipped restoring files, don't try to rollback files
      - rollback not just packages but package dependencies as well
      - pidlock tklbam-backup so it doesn't run more than once
      - chmod 600 the contents of the escrow key (may be sensitive)
      - workaround for webmin getpass.getpass bug on Lucid