Interval INT should backup only if the oldest snapshot in INT-1 is older than INT's period #4

Closed
eddyp opened this Issue Mar 3, 2015 · 0 comments

Comments

Projects
None yet
1 participant
@eddyp
Owner

eddyp commented Mar 3, 2015

When deciding to do a backup for interval INT, there should be a check that the oldest snapshot in INT-1 interval is older than the threshold for the INT interval. Otherwise the INT interval will be populated with backups more frequent than desired and it is possible older backups in INT interval to completely lost.

The condition should be:

ts(oldest(INT-1)) - ts(newest(INT)) >= threshold(INT)

For example:

  • if weekly.0 is from 15th of February and daily.6 is from 17th of February, weekly should not be triggered, but
  • if weekly.0 is from 15th of February and daily.6 is from 23rd of February, weekly shall be triggered

This extra check should probably added in can_backup_interval.

@eddyp eddyp added the bug label Mar 3, 2015

eddyp added a commit that referenced this issue Mar 3, 2015

Fix #4: overdoing backups could lead to backup loss
Do not do a backup in interval INT if the oldest INT-1 or not at
least threshold(INT) seconds newer than the newer snapshot in INT
interval, where threshold(INT) is the the minimum time disance
between 2 backups in the INT interval.

A new extra condition intrdoduced for backing up INT interval:

 ts(oldest(INT-1)) - ts(newest(INT)) >= threshold(INT)

This should ensure backups int INT interval are at least
threshold(INT) seconds apart from each other.

Without this condition previously backups from the INT-1 interval
were promoted indiscriminantly into the INT interval.

This was leading to situations like the one below were weekly
backups would not be at least 1 week apart from each other, or
monthly backups not being at least 30 days from eachother:

drwxr-xr-x 3 eddy eddy 4096 mar  3 21:27 hourly.0
drwxr-xr-x 3 eddy eddy 4096 mar  2 20:38 hourly.1
drwxr-xr-x 3 eddy eddy 4096 mar  1 15:09 daily.0
drwxr-xr-x 3 eddy eddy 4096 feb 27 21:31 daily.1
drwxr-xr-x 3 eddy eddy 4096 feb 26 23:12 daily.2
drwxr-xr-x 3 eddy eddy 4096 feb 25 20:20 daily.3
drwxr-xr-x 3 eddy eddy 4096 feb 25 00:32 daily.4
drwxr-xr-x 3 eddy eddy 4096 feb 24 00:15 daily.5
drwxr-xr-x 3 eddy eddy 4096 feb 15 22:26 weekly.0
drwxr-xr-x 3 eddy eddy 4096 feb 15 22:24 weekly.1
drwxr-xr-x 3 eddy eddy 4096 feb 13 12:12 weekly.2
drwxr-xr-x 3 eddy eddy 4096 feb 13 01:30 weekly.3
drwxr-xr-x 3 eddy eddy 4096 feb  9 22:32 weekly.4
drwxr-xr-x 3 eddy eddy 4096 feb  9 14:21 monthly.0
drwxr-xr-x 3 eddy eddy 4096 ian 26 02:27 monthly.1
drwxr-xr-x 3 eddy eddy 4096 ian 25 22:02 monthly.2
drwxr-xr-x 3 eddy eddy 4096 ian 24 16:50 monthly.3

Note the monthly backups which are actually 1 day apart from
each other (1-2-3 monthly snapshots) and the weekly backups which
are hours apart from each other (see the 0-1 and 2-3 pairs in the
weekly interval).

Signed-off-by: Eddy Petrișor <eddy.petrisor@gmail.com>

eddyp added a commit that referenced this issue Mar 3, 2015

Issue #4: overdoing backups could lead to backup loss
Do not do a backup in interval INT if the oldest INT-1 or not at
least threshold(INT) seconds newer than the newer snapshot in INT
interval, where threshold(INT) is the the minimum time disance
between 2 backups in the INT interval.

A new extra condition intrdoduced for backing up INT interval:

 ts(oldest(INT-1)) - ts(newest(INT)) >= threshold(INT)

This should ensure backups int INT interval are at least
threshold(INT) seconds apart from each other.

This new condition fixes #4.

Without this condition previously backups from the INT-1 interval
were promoted indiscriminantly into the INT interval.

This was leading to situations like the one below were weekly
backups would not be at least 1 week apart from each other, or
monthly backups not being at least 30 days from eachother:

drwxr-xr-x 3 eddy eddy 4096 mar  3 21:27 hourly.0
drwxr-xr-x 3 eddy eddy 4096 mar  2 20:38 hourly.1
drwxr-xr-x 3 eddy eddy 4096 mar  1 15:09 daily.0
drwxr-xr-x 3 eddy eddy 4096 feb 27 21:31 daily.1
drwxr-xr-x 3 eddy eddy 4096 feb 26 23:12 daily.2
drwxr-xr-x 3 eddy eddy 4096 feb 25 20:20 daily.3
drwxr-xr-x 3 eddy eddy 4096 feb 25 00:32 daily.4
drwxr-xr-x 3 eddy eddy 4096 feb 24 00:15 daily.5
drwxr-xr-x 3 eddy eddy 4096 feb 15 22:26 weekly.0
drwxr-xr-x 3 eddy eddy 4096 feb 15 22:24 weekly.1
drwxr-xr-x 3 eddy eddy 4096 feb 13 12:12 weekly.2
drwxr-xr-x 3 eddy eddy 4096 feb 13 01:30 weekly.3
drwxr-xr-x 3 eddy eddy 4096 feb  9 22:32 weekly.4
drwxr-xr-x 3 eddy eddy 4096 feb  9 14:21 monthly.0
drwxr-xr-x 3 eddy eddy 4096 ian 26 02:27 monthly.1
drwxr-xr-x 3 eddy eddy 4096 ian 25 22:02 monthly.2
drwxr-xr-x 3 eddy eddy 4096 ian 24 16:50 monthly.3

Note the monthly backups which are actually 1 day apart from
each other (1-2-3 monthly snapshots) and the weekly backups which
are hours apart from each other (see the 0-1 and 2-3 pairs in the
weekly interval).

Signed-off-by: Eddy Petrișor <eddy.petrisor@gmail.com>

eddyp added a commit that referenced this issue Mar 3, 2015

Issue #4: overdoing backups could lead to backup loss
Do not do a backup in interval INT if the oldest INT-1 is not at
least threshold(INT) seconds newer than the newer snapshot in INT
interval, where threshold(INT) is the the minimum time disance
between 2 backups in the INT interval.

A new extra condition intrdoduced for backing up INT interval:

 ts(oldest(INT-1)) - ts(newest(INT)) >= threshold(INT)

This should ensure backups int INT interval are at least
threshold(INT) seconds apart from each other.

This new condition fixes #4.

Without this condition previously backups from the INT-1 interval
were promoted indiscriminantly into the INT interval.

This was leading to situations like the one below were weekly
backups would not be at least 1 week apart from each other, or
monthly backups not being at least 30 days from eachother:

drwxr-xr-x 3 eddy eddy 4096 mar  3 21:27 hourly.0
drwxr-xr-x 3 eddy eddy 4096 mar  2 20:38 hourly.1
drwxr-xr-x 3 eddy eddy 4096 mar  1 15:09 daily.0
drwxr-xr-x 3 eddy eddy 4096 feb 27 21:31 daily.1
drwxr-xr-x 3 eddy eddy 4096 feb 26 23:12 daily.2
drwxr-xr-x 3 eddy eddy 4096 feb 25 20:20 daily.3
drwxr-xr-x 3 eddy eddy 4096 feb 25 00:32 daily.4
drwxr-xr-x 3 eddy eddy 4096 feb 24 00:15 daily.5
drwxr-xr-x 3 eddy eddy 4096 feb 15 22:26 weekly.0
drwxr-xr-x 3 eddy eddy 4096 feb 15 22:24 weekly.1
drwxr-xr-x 3 eddy eddy 4096 feb 13 12:12 weekly.2
drwxr-xr-x 3 eddy eddy 4096 feb 13 01:30 weekly.3
drwxr-xr-x 3 eddy eddy 4096 feb  9 22:32 weekly.4
drwxr-xr-x 3 eddy eddy 4096 feb  9 14:21 monthly.0
drwxr-xr-x 3 eddy eddy 4096 ian 26 02:27 monthly.1
drwxr-xr-x 3 eddy eddy 4096 ian 25 22:02 monthly.2
drwxr-xr-x 3 eddy eddy 4096 ian 24 16:50 monthly.3

Note the monthly backups which are actually 1 day apart from
each other (1-2-3 monthly snapshots) and the weekly backups which
are hours apart from each other (see the 0-1 and 2-3 pairs in the
weekly interval).

Signed-off-by: Eddy Petrișor <eddy.petrisor@gmail.com>

@eddyp eddyp closed this in 230717d Mar 6, 2015

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment