Skip to content

Commit

Permalink
zwischenstand
Browse files Browse the repository at this point in the history
  • Loading branch information
pstorz committed Nov 24, 2018
1 parent d274582 commit a0b3460
Show file tree
Hide file tree
Showing 42 changed files with 947 additions and 593 deletions.
4 changes: 2 additions & 2 deletions docs/manuals/en/main/general.tex
Expand Up @@ -503,8 +503,8 @@ \section{Terminology}
easily restored. This function is particularly useful if certain
Volumes or Jobs have exceeded their retention period and have been
pruned or purged from the Catalog. Scanning data from Volumes into the
Catalog is done by using the {\bf bscan} program. See the \ilink{bscan
section}{bscan} of the Bareos Utilities chapter of this manual for more
Catalog is done by using the {\bf bscan} program. See the \ilink{bscan section}{bscan}
of the Bareos Utilities chapter of this manual for more
details.

\item [Volume]
Expand Down
33 changes: 29 additions & 4 deletions docs/manuals/en/new_main_reference/replace.sh
Expand Up @@ -70,7 +70,13 @@ sed -i -E 's|:raw-latex:`\\bcommand\{(.*)\}\{(.*)\}`|:command:`\1 \2`|g' *.rst

sed -i -E 's|:raw-latex:`\\file\{(.*)\}`|*\1*|g' *.rst
sed -i -E 's|:raw-latex:`\\directory\{(.*)\}`|*\1*|g' *.rst
#sed -i -E 's#:raw-latex:`\\path\|(.*)\|`#*\1*#g' *.rst

#sed -i -E 's#:raw-latex:`\\path`\\|(.*)\\|#*\1*#g' *.rst
#sed -i -E 's#:raw-latex:`\\path`\\\|([^|])\\\|#\1#g' *.rst

perl -i -pe 's#:raw-latex:`\\path`\\\|(.*?)\\\|(.*)#*\1*#g' *.rst

perl -i -pe 's#:raw-latex:`\\path`\\\|(.*?)\\\|(.*)#*\1*#g' *.rst

sed -i 's/:raw-latex:`\\textbar`/ \\| /g' *.rst
sed -i 's/:raw-latex:`\\textbar{}`/ \\| /g' *.rst
Expand All @@ -82,9 +88,12 @@ sed -i -E 's|:raw-latex:`\\parameter\{(.*)\}`|**\1**|g' *.rst
sed -i -E 's|:raw-latex:`\\argument\{(.*)\}`|**\1**|g' *.rst
sed -i -E 's|:raw-latex:`\\variable\{(.*)\}`|**\1**|g' *.rst

sed -i -E 's#:raw-latex:`\\dt([A-Za-z ]*)`#\1#g' *.rst


sed -i -E 's|:raw-latex:`\\sinceVersion\{(.*)\}\{(.*)\}\{(.*)\}`|\3|g' *.rst


sed -i -E 's|:raw-latex:`\\job\{(.*)\}`|**\1**|g' *.rst
sed -i -E 's|:raw-latex:`\\fileset\{(.*)\}`|**\1**|g' *.rst

Expand All @@ -105,28 +114,44 @@ sed -i -E 's|:raw-latex:`\\trademark\{(.*)\}`|*\1*|g' *.rst
# INDEX
#

sed -i -E 's|:raw-latex:`\\index\[general\]\{(.*)!(.*)\}`|\n.. index:: \n triple: General; \1; \2|g' *.rst
sed -i -E 's|:raw-latex:`\\index\[general\]\{(.*)\}`|.. index:: General; \1 |g' *.rst
#sed -i -E 's|:raw-latex:`\\index\[general\]\{(.*)!(.*)\}`|\n.. index:: \n triple: General; \1; \2|g' *.rst
sed -i -E 's|^( *):raw-latex:`\\index\[general\]\{(.*)!(.*)\}`|\1.. index:: \n triple: General; \2; \3 \n\1.. _`\2`:\n\1|g' *.rst


#sed -i -E 's|:raw-latex:`\\index\[general\]\{(.*)!(.*)\}`|.. _`\1`:\n|g' *.rst

sed -i -E 's|^(.*):raw-latex:`\\index\[general\]\{(.*)\}`|\1.. index:: General; \2 \n\1.. _`\2`:\n\1|g' *.rst
#sed -i -E 's|:raw-latex:`\\index\[general\]\{(.*)\}`|.. _`\1`:|g' *.rst

sed -i -E 's|:raw-latex:`\\index\[sd\]\{(.*)\}`|.. index:: SD; \1 |g' *.rst

#
# LABEL AND REFERENCES
#

sed -i -E 's|:raw-latex:`\\label\{(.*)\}`|.. _\1: \1|g' *.rst
#sed -i -E 's|:raw-latex:`\\label\{(.*)\}`|.. _\1: \1|g' *.rst
sed -i -E 's|:raw-latex:`\\label\{(.*)\}`|\n\n.. _`\1`: \1|g' *.rst

#sed -i -E 's|:raw-latex:`\\nameref\{(.*)\}`|:ref:`\1`|g' *.rst
sed -i -E 's|:raw-latex:`\\nameref\{(.*)\}`|:ref:`\1`|g' *.rst

sed -i -E 's|:raw-latex:`\\elink\{(.*)\}\{(.*)\}`|`\1 <\2>`_|g' *.rst

sed -i -E 's|:raw-latex:`\\ilink\{(.*)\}\{(.*)\}`|:ref:`\2 \1`|g' *.rst
#sed -i -E 's|:raw-latex:`\\ilink\{(.*)\}\{(.*)\}`|:ref:`\1`|g' *.rst

#
# Warnings
#

sed -i -E 's|:raw-latex:`\\warning\{(.*)\}`|\n.. warning:: \n \1|g' *.rst

sed -i -E 's|:raw-latex:`\\linkResourceDirective\{(.*)\}\{(.*)\}\{(.*)\}`|**\3** (\1 configuration, \2 resource)|g' *.rst
sed -i -E 's|:raw-latex:`\\linkResourceDirectiveValue\{(.*)\}\{(.*)\}\{(.*)\}\{(.*)\}`|**\3 = \4** (\1 configuration, \2 resource)|g' *.rst
sed -i -E 's|:raw-latex:`\\resourceDirectiveValue\{(.*)\}\{(.*)\}\{(.*)\}\{(.*)\}`|**\3 = \4** (\1 configuration, \2 resource)|g' *.rst



#
# IMAGES (JUST A FEW, SO MAYBE MANUALLY)
#
Expand Down
78 changes: 41 additions & 37 deletions docs/manuals/en/new_main_reference/source/always-incremental.rst
Expand Up @@ -4,6 +4,8 @@ Always Incremental Backup Scheme
================================

.. index:: General; Always Incremental
.. _`Always Incremental`:


Always Incremental Backups are available since Bareos
16.2.4.
Expand All @@ -12,6 +14,8 @@ Conventional Backup Scheme Drawbacks
------------------------------------

.. index:: General; Retention
.. _`Retention`:


To better understand the advantages of the Always Incremental Backup
scheme, we first analyze the way that the conventional Incremental -
Expand Down Expand Up @@ -66,7 +70,7 @@ These two steps are then executed every day:
- Consolidation of the jobs older than maximum configure age

Deleted files will be in the backup forever, if they are not detected as
deleted using :raw-latex:`\linkResourceDirective{Dir}{Job}{Accurate}`
deleted using **Accurate** (Dir configuration, Job resource)
backup.

The Always Incremental Backup Scheme does not provide the option to have
Expand Down Expand Up @@ -125,23 +129,23 @@ configuration is required:
}
\end{bareosConfigResource}

:raw-latex:`\linkResourceDirectiveValue{Dir}{Job}{Accurate}{yes}`
**Accurate = yes** (Dir configuration, Job resource)
is required to detect deleted files and prevent that they are kept
in the consolidated backup jobs.

:raw-latex:`\linkResourceDirectiveValue{Dir}{Job}{Always Incremental}{yes}`
**Always Incremental = yes** (Dir configuration, Job resource)
enables the Always Incremental feature.

:raw-latex:`\linkResourceDirective{Dir}{Job}{Always Incremental Job Retention}`
**Always Incremental Job Retention** (Dir configuration, Job resource)
set the age where incrementals of this job will be kept, older jobs
will be consolidated.

:raw-latex:`\linkResourceDirective{Dir}{Job}{Always Incremental Keep Number}`
**Always Incremental Keep Number** (Dir configuration, Job resource)
sets the number of incrementals that will be kept without regarding
the age. This should make sure that a certain history of a job will
be kept even if the job is not executed for some time.

:raw-latex:`\linkResourceDirective{Dir}{Job}{Always Incremental Max Full Age}`
**Always Incremental Max Full Age** (Dir configuration, Job resource)
is described later, see
:ref:`sec:AlwaysIncrementalMaxFullAge`.

Expand All @@ -159,22 +163,22 @@ Consolidate Job
}
\end{bareosConfigResource}

:raw-latex:`\resourceDirectiveValue{Dir}{Job}{Type}{Consolidate}`
**Type = Consolidate** (Dir configuration, Job resource)
configures a job to be a consolidate job. This type have been
introduced with the Always Incremental feature. When used, it
automatically trigger the consolidation of incremental jobs that
need to be consolidated.

:raw-latex:`\linkResourceDirectiveValue{Dir}{Job}{Accurate}{yes}`
**Accurate = yes** (Dir configuration, Job resource)
let the generated virtual backup job keep the accurate information.

:raw-latex:`\linkResourceDirective{Dir}{Job}{Max Full Consolidations}`
**Max Full Consolidations** (Dir configuration, Job resource)
is described later, see
:ref:`sec:MaxFullConsolidations`.

The **Consolidate** job evaluates all jobs configured
with
:raw-latex:`\linkResourceDirectiveValue{Dir}{Job}{Always Incremental}{yes}`.
**Always Incremental = yes** (Dir configuration, Job resource).
When a job is selected for consolidation, all job runs are taken into
account, independent of the pool and storage where they are located.

Expand Down Expand Up @@ -296,13 +300,13 @@ The following image shows the available backups for each day:
- Every day a incremental backup is done and is additionally available.

- When the age of the oldest incremental reaches
:raw-latex:`\linkResourceDirective{Dir}{Job}{Always Incremental Job Retention}`,
**Always Incremental Job Retention** (Dir configuration, Job resource),
the consolidation job consolidates the oldest incremental with the
full backup before to a new full backup.

This can go on more or less forever and there will be always an
incremental history of
:raw-latex:`\linkResourceDirective{Dir}{Job}{Always Incremental Job Retention}`.
**Always Incremental Job Retention** (Dir configuration, Job resource).

The following plot shows what happens if a job is not run for a certain
amount of time.
Expand All @@ -314,18 +318,18 @@ until the last incremental is too old and then only one full backup is
left. This is usually not what is intended.

For this reason, the directive
:raw-latex:`\linkResourceDirective{Dir}{Job}{Always Incremental Keep Number}`
**Always Incremental Keep Number** (Dir configuration, Job resource)
is available which sets the minimum number of incrementals that should
be kept even if they are older than
:raw-latex:`\linkResourceDirective{Dir}{Job}{Always Incremental Job Retention}`.
**Always Incremental Job Retention** (Dir configuration, Job resource).

Setting
:raw-latex:`\linkResourceDirective{Dir}{Job}{Always Incremental Keep Number}`
**Always Incremental Keep Number** (Dir configuration, Job resource)
to 7 in our case leads to the following result:

|image|

:raw-latex:`\linkResourceDirective{Dir}{Job}{Always Incremental Keep Number}`
**Always Incremental Keep Number** (Dir configuration, Job resource)
incrementals are always kept, and when the backup starts again the
consolidation of old incrementals starts again.

Expand Down Expand Up @@ -377,29 +381,29 @@ Always Incremental Max Full Age
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

To be able to cope with this problem, the directive
:raw-latex:`\linkResourceDirective{Dir}{Job}{Always Incremental Max Full Age}`
**Always Incremental Max Full Age** (Dir configuration, Job resource)
was added. When
:raw-latex:`\linkResourceDirective{Dir}{Job}{Always Incremental Max Full Age}`
**Always Incremental Max Full Age** (Dir configuration, Job resource)
is configured, in daily operation the Full Backup is left untouched
while the incrementals are consolidated as usual. Only if the Full
Backup is older than
:raw-latex:`\linkResourceDirective{Dir}{Job}{Always Incremental Max Full Age}`,
**Always Incremental Max Full Age** (Dir configuration, Job resource),
the full backup will also be part of the consolidation.

Depending on the setting of the
:raw-latex:`\linkResourceDirective{Dir}{Job}{Always Incremental Max Full Age}`,
**Always Incremental Max Full Age** (Dir configuration, Job resource),
the amount of daily data being moved can be reduced without losing the
advantages of the always incremental Backup Scheme.

:raw-latex:`\linkResourceDirective{Dir}{Job}{Always Incremental Max Full Age}`
**Always Incremental Max Full Age** (Dir configuration, Job resource)
must be larger than
:raw-latex:`\linkResourceDirective{Dir}{Job}{Always Incremental Job Retention}`.
**Always Incremental Job Retention** (Dir configuration, Job resource).

The resulting interval between full consolidations when running daily
backups and daily consolidations is
:raw-latex:`\linkResourceDirective{Dir}{Job}{Always Incremental Max Full Age}`
**Always Incremental Max Full Age** (Dir configuration, Job resource)
-
:raw-latex:`\linkResourceDirective{Dir}{Job}{Always Incremental Job Retention}`.
**Always Incremental Job Retention** (Dir configuration, Job resource).

.. raw:: latex

Expand All @@ -425,10 +429,10 @@ Max Full Consolidations
~~~~~~~~~~~~~~~~~~~~~~~

When the
:raw-latex:`\linkResourceDirective{Dir}{Job}{Always Incremental Max Full Age}`
**Always Incremental Max Full Age** (Dir configuration, Job resource)
of many clients is set to the same value, it is probable that all full
backups will reach the
:raw-latex:`\linkResourceDirective{Dir}{Job}{Always Incremental Max Full Age}`
**Always Incremental Max Full Age** (Dir configuration, Job resource)
at once and so consolidation jobs including the full backup will be
started for all clients at once. This would again mean that the whole
data being stored from all clients will be moved in one day.
Expand All @@ -443,12 +447,12 @@ As can be seen, virtual jobs including the full are triggered for all
three clients at the same time.

This is of course not desirable so the directive
:raw-latex:`\linkResourceDirective{Dir}{Job}{Max Full Consolidations}`
**Max Full Consolidations** (Dir configuration, Job resource)
was introduced.

:raw-latex:`\linkResourceDirective{Dir}{Job}{Max Full Consolidations}`
**Max Full Consolidations** (Dir configuration, Job resource)
needs to be configured in the
:raw-latex:`\resourceDirectiveValue{Dir}{Job}{Type}{Consolidate}` job:
**Type = Consolidate** (Dir configuration, Job resource) job:

.. raw:: latex

Expand All @@ -464,7 +468,7 @@ needs to be configured in the
\end{bareosConfigResource}

If
:raw-latex:`\linkResourceDirective{Dir}{Job}{Max Full Consolidations}`
**Max Full Consolidations** (Dir configuration, Job resource)
is configured, the consolidation job will not start more than the
specified Consolidations that include the Full Backup.

Expand All @@ -474,9 +478,9 @@ jobs are completed before the next normal backup run starts.

The number of always incremental jobs, the interval that the jobs are
triggered and the setting of
:raw-latex:`\linkResourceDirective{Dir}{Job}{Always Incremental Max Full Age}`
**Always Incremental Max Full Age** (Dir configuration, Job resource)
influence the value that makes sense for
:raw-latex:`\linkResourceDirective{Dir}{Job}{Max Full Consolidations}`.
**Max Full Consolidations** (Dir configuration, Job resource).

.. raw:: latex

Expand Down Expand Up @@ -507,7 +511,7 @@ store a certain job for a longer time.

When using always incremental, the usual maximum age of data stored
during the backup cycle is
:raw-latex:`\linkResourceDirective{Dir}{Job}{Always Incremental Job Retention}`.
**Always Incremental Job Retention** (Dir configuration, Job resource).

Usually, it is desired to be able to store a certain backup for a longer
time, e.g. monthly a backup should be kept for half a year.
Expand Down Expand Up @@ -576,14 +580,14 @@ the data should be stored in a long-term pool.
\end{bareosConfigResource}

To make sure the longterm
:raw-latex:`\resourceDirectiveValue{Dir}{Job}{Level}{VirtualFull}` is
**Level = VirtualFull** (Dir configuration, Job resource) is
not taken as base for the next incrementals, the job type of the copied
job is set to
:raw-latex:`\resourceDirectiveValue{Dir}{Job}{Type}{Archive}` with the
:raw-latex:`\linkResourceDirective{Dir}{Job}{Run Script}`.
**Type = Archive** (Dir configuration, Job resource) with the
**Run Script** (Dir configuration, Job resource).

As can be seen on the plot, the
:raw-latex:`\resourceDirectiveValue{Dir}{Job}{Level}{VirtualFull}`
**Level = VirtualFull** (Dir configuration, Job resource)
archives the current data, i.e. it consolidates the full and all
incrementals that are currently available.

Expand Down

0 comments on commit a0b3460

Please sign in to comment.