Minio Rolling Backup Management Container
This container provides a clean, job based, approach to rotating backups
on a schedule according to given rules. It is intended to be used in
conjunction with the various
minback containers which place backup
files in a Minio bucket for you.
- Allows multiple backup retention periods to be selected
- Lightweight and short lived
- Simple and well tested Go implementation
In this example, let's assume we have a very aggressive backup schedule that creates a new backup every minute. We would like to keep:
- Every backup for the last
- A backup every
- A backup every
- A backup every week (
7d) after 1 week
- A backup every 4 weeks (
4w) after a month (
- A backup every year after 3 years (
docker run --rm --env-file backup.env minback/cleanup cleanup --db my_db --keep "@1h/15m~5m" --keep "@6h/1h~15m" --keep "@7d/1w~1d" --keep "@4w/4w~1w" --keep "@52w/156w~1w"
MINIO_SERVER=https://play.minio.io MINIO_ACCESS_KEY=minio MINIO_SECRET_KEY=miniosecret MINIO_BUCKET=backups
A keep specification is composed of three different properties:
@) tells us when this window starts to apply from. It defaults to "right now" and can be extended out with a time duration, for example
52wtells us that the window starts at 1 year and continues into the past from there.
Windows "stack", so
@4wwill combine to have one specification applying from
52wand the next from
/) tells us which backups within a window to keep. A spacing of
15mwill keep any backups which fall on the
15mboundary (according to their timestamp) - so
Smudge Factor (
~) tells us that we should round the backup's time to the nearest provided duration. This is great for situations where your backup's timestamp is a lot more accurate than your window and you wish to smudge things to ensure it falls on a boundary.
For example, a smudge of
11:15:00and have it match the
/15mspacing we selected previously. You can also use larger smudges like
~52w(for a year), but keep in mind that we don't de-duplicate things if multiple backups match because of the same smudge factor.
You define a keep specification by concatenating each property's control character with its duration and listing them in any order you wish.
In this example, let's assume we create a new backup every 6 hours but only want to store one backup per day for everything older than 2 weeks (i.e. your 6 hour frequency is for short-term recovery).
- Window is 2 weeks (
2w), meaning this specification will only apply to backups older than 2 weeks, and until an older window is found.
- Spacing is 1 day (
1d), meaning we will keep one backup per day within this window. Backups will be selected based on their timestamp and only those which match (after smudging) the day boundary will be kept.
- Smudge Factor is 6 hours (
6h) which ensures that backups which occur up to 3 hours either side of the day boundary (midnight) will be kept.
NAME: main.exe - A new cli application USAGE: main.exe [global options] command [command options] [arguments...] VERSION: v1.0.0 AUTHOR: Benjamin Pannell <email@example.com> COMMANDS: cleanup help, h Shows a list of commands or help for one command GLOBAL OPTIONS: --log-level value DEBUG|INFO|WARN|ERROR (default: "INFO") --help, -h show help --version, -v print the version COPYRIGHT: Sierra Softworks © 2017
NAME: main.exe cleanup - USAGE: main.exe cleanup [command options] [arguments...] OPTIONS: --server value [$MINIO_SERVER] --access-key value [$MINIO_ACCESS_KEY] --secret-key value [$MINIO_SECRET_KEY] --bucket value (default: "backups") [$MINIO_BUCKET] --db value The name of the database backup files (my-db-2017-12-19.backup would use 'my-db') --keep value, -k value @7d/1d will keep a backup every 1d for all backups 7d old or older
You can configure command line options using environment variables if you wish.
The Minio server you wish to send backups to.
The Access Key used to connect to your Minio server.
The Secret Key used to connect to your Minio server.
The Minio bucket you wish to store your backup in.