Skip to content

Latest commit

 

History

History
341 lines (202 loc) · 8.23 KB

rmexp.pod

File metadata and controls

341 lines (202 loc) · 8.23 KB
Raw

NAME

rmexp - a Perl script set for automatic removal of expired backups

  • rmexp-local removes backups stored as files on local file system

  • rmexp-smb removes backups located at SMB shares

  • rmexp-zfs-snap removes backups stored as ZFS snapshots

SYNOPSIS

rmexp-local|rmexp-smb|rmexp-zfs-snap [-a] [-d] [-m] [-n] [-s] -c configfile

rmexp-local|rmexp-smb|rmexp-zfs-snap -l [-m] [-s] -c configfile

rmexp-local|rmexp-smb|rmexp-zfs-snap --help|-?|--man

Options:
      -a               include all available backups in the report
      -d               print performed actions
      -c               configuration file name
      -l               list backups and exit
      -m               mount/unmount file system (rmexp-local only)
      -n               dry run
      -s               suppress report if no warnings

      --ignore_written ignore written check (rmexp-zfs-snap only)

      --help -?         brief help message
      --man             full documentation

DESCRIPTION

Overview

Rmexp is a Perl script set for automatic removal of expired backups. Rmexp uses a BackupPC-like expiration algorithm.

Rmexp can be configured to keep a certain number of backups. Exponential expiry is also supported, allowing backups with various vintages to be kept (for example, a settable number of most recent weekly backups, plus a settable number of older backups that are 2, 4, 8, or 16 weeks apart).

Features:

  • The standard smb protocol could be used to access backup data.

  • Prints on STDOUT action report and current state of the backup directory. If some backups are outdated warning message will be prepended to the report.

CONFIGURATION

Backup file/snapshot name syntax and parsing

{prefix}_YYYY-mm-dd_HH-MM-SS[.level]{suffix}

e.g. mydump_2010-02-10_16-48-23.dump.gz

or mydump_2010-02-10_16-48-23.0.dump.gz

Prefix is arbitrary, but rmexp uses it for backups grouping. Rmexp process backups separately for each prefix.

"Level" is a level of this backup. A full backup has level 0. Incrementals are 1. Multi-level incrementals are not supported, so this is just a 0 or 1. Level is optional for full backups.

$Conf{fileNameSuffix}

is a PCRE without anchors. Suffix part of the file/snapshot name should match $Conf{fileNameSuffix}.

Example: '\.dump\.gz'.

Rmexp assumes all files/snapshots with suffix that matches $Conf{fileNameSuffix} should have name syntax matches $Conf{fileNameRE}. If file/snapshot name matches $Conf{fileNameSuffix} but does not match $Conf{fileNameRE} then rmexp ignores that file and throws warning. Rmexp ignores backup names with other suffixes (e.g. logs, descriptions or so on) silently.

$Conf{fileNameRE}

is a PCRE without anchors for parsing file/snapshot name. It should have capturing parenthesises:

$1  prefix
$2  YYYY
$3  mm
$4  dd
$5  HH
$6  MM
$7  SS
$8  level

Example:

qr/^(.+)_
  (\d{4})-(\d{1,2})-(\d{1,2})_(\d{1,2})-(\d{1,2})-(\d{1,2})
  (?:\.(\d))?/x,

Local file system specific settings

$Conf{backupDir}

Directory where backups are stored.

$Conf{maxDepth}

Default is 255.

Rmexp recursively searches all directories below $Conf{backupDir}. If directory depth exceeds this limit rmexp dies with fatal error.

$Conf{mountPoint}

The mount point to mount a file system to if -m option is enabled.

Rmexp will try to mount a file system to the specified mount point prior doing anything and unmount it (if it was mounted by Rmexp) after operations that require access to the backup directory are completed.

The mount point should be configured in /etc/fstab with noauto option, for instance:

//backup@server/share  /smb/server/share  smbfs  rw,noauto,-I=192.168.1.1  0  0

ZFS specific settings

$Conf{zfsFS}

ZFS file system that should be processed.

$Conf{ignoreSnapshotPrefix}

Ignore snapshots with a given name prefix.

SMB specific settings

$Conf{SmbShareUserName}
$Conf{SmbSharePasswd}

Username and password to access SMB share.

$Conf{SmbHostName}

Name or IP address of the SMB server.

$Conf{SmbShareName}

Name of the SMB share.

$Conf{SmbDir}

Path (excluding share name) to the backups directory.

$Conf{maxDepth}

Default is 255.

Rmexp recursively searches all directories below $Conf{SmbDir}. If directory depth exceeds this limit rmexp dies with fatal error.

What to remove and when to do it

Rmexp uses BackupPC code for backups aging. For detailed algorithm and options description refer to "What to backup and when to do it" section of the BackupPC 4.0.0 documentation. http://backuppc.sourceforge.net

$Conf{FullPeriod} = 1;

Minimum period in days between backups.

$Conf{IncrPeriod} = -1;

Minimum period in days between incremental backups. Not affects backups aging. If the last backup is older than $Conf{IncrPeriod} then outdated backup warning will be added to statistics output. If $Conf{IncrPeriod} is undefined or set below or equal to 0, then $Conf{FullPeriod} will be used instead.

$Conf{FillCycle} = 0;
$Conf{FullKeepCnt} = 1;

Number of backups to keep. Must be >= 1.

Exponential expiry is specified using an array for $Conf{FullKeepCnt}:

$Conf{FullKeepCnt} = [4, 2, 3];
$Conf{FullKeepCntMin} = 1;
$Conf{FullAgeMax} = 90;

Very old backups are removed after $Conf{FullAgeMax} days. However, we keep at least $Conf{FullKeepCntMin} backups no matter how old they are.

$Conf{IncrKeepCnt} = 6;

Number of incremental backups to keep. Must be >= 1.

$Conf{IncrKeepCntMin} = 1;
$Conf{IncrAgeMax} = 30;

Very old incremental backups are removed after $Conf{IncrAgeMax} days. However, we keep at least $Conf{IncrKeepCntMin} incremental backups no matter how old they are.

REQUIREMENTS

  • A Linux, Solaris, or Unix based server.

  • Perl version 5.8.0 or later. If you don't have perl, please see http://www.cpan.org.

  • Filesys::SmbClient Perl module for access SMB shares.

INSTALLATION

  1. Place rmexp directory where you want.

  2. Edit values in configuration file to match your needs.

  3. Add entries to crontab.

EXAMPLES

Crontab entries example:

45 7 * * 2-5 /somedir/rmexp-local -s -c /somedir/rmexp.conf | mail -Es "rmexp@`hostname` `date '+\%Y-\%m-\%d \%H:\%M'` backup warning" mbox@domain.tld

45 7 * * 6 /somedir/rmexp-local -c /somedir/rmexp.conf | mail -s "rmexp@`hostname` `date '+\%Y-\%m-\%d \%H:\%M'` weekly report" mbox@domain.tld

In case of a fatal error rmexp may not write any output to STDOUT. As a result, cron will send a mail message with an empty body. It is possible to copy STDERR to STDOUT and keep STDERR (as we still need to write errors in the cron log) using the following construct:

(( my_command 3>&1 1>&2 2>&3 3>- | tee /dev/stderr ) 3>&1 1>&2 2>&3 3>- ) | mail ...

Here is a full example:

45 7 * * * (( /somedir/rmexp-local -m -c /somedir/rmexp.conf 3>&1 1>&2 2>&3 3>- | tee /dev/stderr ) 3>&1 1>&2 2>&3 3>- ) | mail -s "rmexp@`hostname` `date '+\%Y-\%m-\%d \%H:\%M'` daily report" mbox@domain.tld

AUTHOR

Alexander Moisseev <moiseev@mezonplus.ru>

USED / MODIFIED SOURCE CODE

Subroutines from BackupPC_dump by Craig Barratt <cbarratt@users.sourceforge.net>

LICENSE and COPYRIGHT

Copyright (C) 2011-2017 Alexander Moisseev
All rights reserved.

This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License in the LICENSE file along with this program; if not, write to the Free Software Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA.