Skip to content
This repository has been archived by the owner on Mar 19, 2021. It is now read-only.

Commit

Permalink
cleanup Maximum Concurrent Jobs and Recycle
Browse files Browse the repository at this point in the history
adding description of multi device configuration for running concurrent
jobs on disk storage.
  • Loading branch information
joergsteffens committed Jun 2, 2016
1 parent 023a8da commit c03310b
Show file tree
Hide file tree
Showing 16 changed files with 487 additions and 800 deletions.
38 changes: 10 additions & 28 deletions manuals/en/main/bareos-fd-resource-client-definitions.tex
Expand Up @@ -45,7 +45,7 @@
If runscripts are not needed it would be recommended as a security measure to disable
running those or only allow the commands that you really want to be used.

Runscripts are particularly a problem as they allow the filedaemon to run
Runscripts are particularly a problem as they allow the \bareosFd to run
arbitrary commands. You may also look into the \linkResourceDirective{Fd}{Client}{Allowed Script Dir} keyword to
limit the impact of the runscript command.
}
Expand Down Expand Up @@ -115,10 +115,8 @@
}

\defDirective{Fd}{Client}{FD Source Address}{}{}{%
This record is optional, and if it is specified, it will cause the File
daemon server (for Storage connections) to bind to the specified {\bf
IP-Address}, which is either a domain name or an IP address specified as a
dotted quadruple. If this record is not specified, the kernel will choose
If specified, the \bareosFd will bind to the specified address when creating outbound connections.
If this record is not specified, the kernel will choose
the best address according to the routing table (the default).
}

Expand All @@ -133,44 +131,28 @@
that does not follow Internet standards and times out a valid
connection after a short duration despite the fact that keepalive is
set. This usually results in a broken pipe error message.

% If you continue getting broken pipe error messages despite using the
% Heartbeat Interval, and you are using Windows, you should consider
% upgrading your ethernet driver. This is a known problem with NVidia
% NForce 3 drivers (4.4.2 17/05/2004), or try the following workaround
% suggested by Thomas Simmons for Win32 machines:
%
% Browse to:
% Start {\textgreater} Control Panel {\textgreater} Network Connections
%
% Right click the connection for the nvidia adapter and select properties.
% Under the General tab, click "Configure...". Under the Advanced tab set
% "Checksum Offload" to disabled and click OK to save the change.

% Lack of communications, or communications that get interrupted can
% also be caused by Linux firewalls where you have a rule that throttles
% connections or traffic.
}

\defDirective{Fd}{Client}{LMDB Threshold}{}{}{%
}

\defDirective{Fd}{Client}{Maximum Bandwidth Per Job}{}{}{%
The speed parameter specifies the maximum allowed bandwidth that a job may
use. The speed parameter should be specified in k/s, kb/s, m/s or mb/s.
use.
}

\defDirective{Fd}{Client}{Maximum Concurrent Jobs}{}{}{%
where {\textless}number{\textgreater} is the maximum number of Jobs that should run
This directive specifies the maximum number of Jobs that should run
concurrently. Each contact from the Director (e.g. status request, job start
request) is considered as a Job, so if you want to be able to do a {\bf
status} request in the console at the same time as a Job is running, you
request) is considered as a Job,
so if you want to be able to do a \bcommand{status} request in the console
at the same time as a Job is running, you
will need to set this value greater than 1.
}

\defDirective{Fd}{Client}{Maximum Network Buffer Size}{}{}{%
where {\textless}bytes{\textgreater} specifies the initial network buffer size to use with
the File daemon. This size will be adjusted down if it is too large until it
This directive specifies the initial network buffer size to use.
This size will be adjusted down if it is too large until it
is accepted by the OS. Please use care in setting this value since if it is
too large, it will be trimmed by 512 bytes until the OS is happy, which may
require a large number of system calls. The default value is 65,536 bytes.
Expand Down
2 changes: 1 addition & 1 deletion manuals/en/main/bareos-manual-main-reference.tex
Expand Up @@ -187,7 +187,7 @@ \part{Tasks and Concepts}
\include{restore}

\chapter{Volume Management}
\include{disk}
\input{disk}
\include{recycling}

\include{pools}
Expand Down
14 changes: 6 additions & 8 deletions manuals/en/main/bareos-sd-resource-storage-definitions.tex
Expand Up @@ -69,17 +69,15 @@
}

\defDirective{Sd}{Storage}{Maximum Concurrent Jobs}{}{}{%
where {\textless}number{\textgreater} is the maximum number of Jobs that may run
concurrently. The default is set to 10, but you may set it to a larger
number. Each contact from the Director (e.g. status request, job start
request) is considered as a Job, so if you want to be able to do a {\bf
status} request in the console at the same time as a Job is running, you
This directive specifies the maximum number of Jobs that may run
concurrently. Each contact from the Director (e.g. status request, job start
request) is considered as a Job, so if you want to be able to do a \bcommand{status}{}
request in the console at the same time as a Job is running, you
will need to set this value greater than 1. To run simultaneous Jobs,
you will need to set a number of other directives in the Director's
configuration file. Which ones you set depend on what you want, but you
will almost certainly need to set the {\bf Maximum Concurrent Jobs} in
the Storage resource in the Director's configuration file and possibly
those in the Job and Client resources.
will almost certainly need to set the \linkResourceDirective{Dir}{Storage}{Maximum Concurrent Jobs}.
Please refer to the \nameref{ConcurrentJobs} chapter.
}

\defDirective{Sd}{Storage}{Maximum Network Buffer Size}{}{}{%
Expand Down
11 changes: 9 additions & 2 deletions manuals/en/main/bareos.sty
Expand Up @@ -182,12 +182,19 @@
}
\newcommand{\parameter}[1]{\path|#1|}
\newcommand{\pluginevent}[1]{\path|#1|}
\newcommand{\pool}[1]{\path|#1|}
\newcommand{\pool}[1]{\resourcename{Dir}{Pool}{#1}}
\newcommand{\argument}[1]{\textit{#1}}
\newcommand{\resourcename}[1]{\path|#1|}
\newcommand{\resourcetype}[2]{\path|#2|$^{\mbox{\tiny #1}}$}
\newcommand{\resourcename}[3]{\path|#3|$^{\mbox{\tiny #1}}_{\mbox{\tiny #2}}$}
\newcommand{\registrykey}[1]{\path|#1|}
\newcommand{\variable}[1]{\path|#1|}
\newcommand{\volume}[1]{\path|#1|}
\newcommand{\volumestatus}[1]{\path|#1|}
\newcommand{\volumeparameter}[2]{\ifthenelse{\isempty{#2}}{%
\path|#1|%
}{%
\path|#1 = #2|%
}}
\newcommand{\os}[2]{\ifthenelse{\isempty{#2}}{%
\path|#1|\index[general]{Platform!#1}%
}{%
Expand Down
13 changes: 5 additions & 8 deletions manuals/en/main/bconsole.tex
Expand Up @@ -326,7 +326,6 @@ \section{Console Keywords}
\end{description}

\section{Console Commands}
\index[general]{Console!Commands}
\label{sec:ConsoleCommands}

The following commands are currently implemented:
Expand Down Expand Up @@ -1687,8 +1686,7 @@ \section{Console Commands}

\subsection{Special dot (.) Commands}
\label{dotcommands}
\index[general]{Console!Command!Special .}
\index[general]{Console!Command!. Commands}
\index[general]{Console!Command!. commands}

There is a list of commands that are prefixed with a period (.). These
commands are intended to be used either by batch programs or graphical user
Expand All @@ -1712,8 +1710,8 @@ \subsection{Special At (@) Commands}
\index[general]{Console!Command!\at{}input {\textless}filename{\textgreater}}
Read and execute the commands contained in the file specified.

\item [@output {\textless}filename{\textgreater} w/a]
\index[general]{Console!Command!\at{}output {\textless}filename{\textgreater} w/a}
\item [@output {\textless}filename{\textgreater} {\textless}w{\textbar}a{\textgreater}]
\index[general]{Console!Command!\at{}output {\textless}filename{\textgreater} {\textless}w{\textbar}a{\textgreater}}
Send all following output to the filename specified either overwriting the
file (w) or appending to the file (a). To redirect the output to the
terminal, simply enter {\bf @output} without a filename specification.
Expand All @@ -1725,12 +1723,11 @@ \subsection{Special At (@) Commands}
@output /dev/null
commands ...
@output
\end{verbatim}
\normalsize

\item [@tee {\textless}filename{\textgreater} w/a]
\index[general]{Console!Command!\at{}tee {\textless}filename{\textgreater} w/a}
\item [@tee {\textless}filename{\textgreater} {\textless}w{\textbar}a{\textgreater}]
\index[general]{Console!Command!\at{}tee {\textless}filename{\textgreater} {\textless}w{\textbar}a{\textgreater}}
Send all subsequent output to both the specified file and the terminal. It is
turned off by specifying {\bf @tee} or {\bf @output} without a filename.

Expand Down
13 changes: 6 additions & 7 deletions manuals/en/main/director-resource-client-definitions.tex
Expand Up @@ -91,7 +91,7 @@
\defDirective{Dir}{Client}{Job Retention}{}{}{%
The Job Retention directive defines the length of time that Bareos will keep
Job records in the Catalog database after the Job End time. When
this time period expires, and if {\bf AutoPrune} is set to {\bf yes}
this time period expires, and if \linkResourceDirective{Dir}{Client}{Auto Prune} is set to {\bf yes}
Bareos will prune (remove) Job records that are older than the specified
File Retention period. As with the other retention periods, this
affects only records in the catalog and not data in your archive backup.
Expand All @@ -100,14 +100,14 @@
records will also be pruned regardless of the File Retention period set.
As a consequence, you normally will set the File retention period to be
less than the Job retention period. The Job retention period can actually
be less than the value you specify here if you set the {\bf Volume
Retention} directive in the Pool resource to a smaller duration. This is
be less than the value you specify here if you set the \linkResourceDirective{Dir}{Pool}{Volume
Retention} directive to a smaller duration. This is
because the Job retention period and the Volume retention period are
independently applied, so the smaller of the two takes precedence.

The Job retention period is specified as seconds, minutes, hours, days,
weeks, months, quarters, or years. See the
\ilink{ Configuration chapter}{Time} of this manual for
\ilink{Configuration chapter}{Time} of this manual for
additional details of time specification.

The default is 180 days.
Expand All @@ -120,12 +120,11 @@
}

\defDirective{Dir}{Client}{Maximum Concurrent Jobs}{}{}{%
where {\textless}number{\textgreater} is the maximum number of Jobs with the current Client
This directive specifies the maximum number of Jobs with the current Client
that can run concurrently. Note, this directive limits only Jobs for Clients
with the same name as the resource in which it appears. Any other
restrictions on the maximum concurrent jobs such as in the Director, Job, or
restrictions on the maximum concurrent jobs such as in the Director, Job or
Storage resources will also apply in addition to any limit specified here.
The default is set to 1, but you may set it to a larger number.
}

\defDirective{Dir}{Client}{Name}{}{}{%
Expand Down
11 changes: 4 additions & 7 deletions manuals/en/main/director-resource-director-definitions.tex
Expand Up @@ -85,12 +85,10 @@
}

\defDirective{Dir}{Director}{Maximum Concurrent Jobs}{}{}{%
\label{DirMaxConJobs}%
\index[general]{Simultaneous Jobs}%
\index[general]{Concurrent Jobs}%
where {\textless}number{\textgreater} is the maximum number of total Director Jobs that
should run concurrently. The default is set to 1, but you may set it to a
larger number.
This directive specifies the maximum number of total Director Jobs that
should run concurrently.

The Volume format becomes more complicated with
multiple simultaneous jobs, consequently, restores may take longer if
Expand All @@ -104,9 +102,8 @@
}

\defDirective{Dir}{Director}{Maximum Console Connections}{}{}{%
where \parameter{number} is the maximum number of Console Connections that
could run concurrently. The default is set to 20, but you may set it to a
larger number.
This directive specifies the maximum number of Console Connections that
could run concurrently.
}

\defDirective{Dir}{Director}{Messages}{}{}{%
Expand Down
12 changes: 6 additions & 6 deletions manuals/en/main/director-resource-job-definitions.tex
Expand Up @@ -560,8 +560,8 @@
from when the job starts, ({\bf not} necessarily the same as when the
job was scheduled).

By default, the the watchdog thread will kill any Job that has run more
than 6 days. The maximum watchdog timeout is independent of MaxRunTime
By default, the watchdog thread will kill any Job that has run more
than 6 days. The maximum watchdog timeout is independent of \configdirective{Max Run Time}
and cannot be changed.
}

Expand Down Expand Up @@ -610,10 +610,10 @@
Job resource that can run concurrently. Note, this directive limits
only Jobs with the same name as the resource in which it appears. Any
other restrictions on the maximum concurrent jobs such as in the
Director, Client, or Storage resources will also apply in addition to
the limit specified here. The default is set to 1, but you may set it
to a larger number. We strongly recommend that you read the WARNING
documented under \nameref{DirMaxConJobs}.
Director, Client or Storage resources will also apply in addition to
the limit specified here.

For details, see the \nameref{ConcurrentJobs} chapter.
}

\defDirective{Dir}{Job}{Maxrun Sched Time}{}{}{%
Expand Down
18 changes: 9 additions & 9 deletions manuals/en/main/director-resource-pool-definitions.tex
Expand Up @@ -121,8 +121,8 @@
\configdirective{Label Format = "File-"},
the first volumes will be named \volume{File-0001}, \volume{File-0002}, ...
With the exception of Job specific variables, you can test your
\configdirective{Label Format}
With the exception of Job specific variables, you can test your
\configdirective{Label Format}
by using the \ilink{var command}{var} the Console Chapter
of this manual.
Expand Down Expand Up @@ -300,17 +300,17 @@
proper retention periods. However, by using this option you risk losing
valuable data.
Please be aware that {\bf Purge Oldest Volume} disregards all retention
\warning{
Be aware that \configdirective{Purge Oldest Volume} disregards all retention
periods. If you have only a single Volume defined and you turn this
variable on, that Volume will always be immediately overwritten when it
fills! So at a minimum, ensure that you have a decent number of Volumes
in your Pool before running any jobs. If you want retention periods to
apply do not use this directive. To specify a retention period, use the
{\bf Volume Retention} directive (see above).
We {\bf highly} recommend against using this directive, because it is
sure that some day, Bareos will recycle a Volume that contains current
data. The default is {\bf no}.
apply do not use this directive.\\
We \textbf{highly} recommend against using this directive, because it is
sure that some day, Bareos will purge a Volume that contains current
data.
}
}
\defDirective{Dir}{Pool}{Recycle}{}{}{%
Expand Down
18 changes: 8 additions & 10 deletions manuals/en/main/director-resource-storage-definitions.tex
Expand Up @@ -87,21 +87,19 @@
}

\defDirective{Dir}{Storage}{Maximum Concurrent Jobs}{}{}{%
where {\textless}number{\textgreater} is the maximum number of Jobs with the current
This directive specifies the maximum number of Jobs with the current
Storage resource that can run concurrently. Note, this directive limits
only Jobs for Jobs using this Storage daemon. Any other restrictions on
the maximum concurrent jobs such as in the Director, Job, or Client
resources will also apply in addition to any limit specified here. The
default is set to 1, but you may set it to a larger number. However, if
you set the Storage daemon's number of concurrent jobs greater than one,
we recommend that you read the waring documented under \ilink{Maximum
Concurrent Jobs}{DirMaxConJobs} in the Director's resource or simply
turn data spooling on as documented in the \ilink{Data
Spooling}{SpoolingChapter} chapter of this manual.
the maximum concurrent jobs such as in the Director, Job or Client
resources will also apply in addition to any limit specified here.

If you set the Storage daemon's number of concurrent jobs greater than one,
we recommend that you read \nameref{ConcurrentJobs} and/or
turn data spooling on as documented in \nameref{SpoolingChapter}.
}

\defDirective{Dir}{Storage}{Maximum Concurrent Read Jobs}{}{}{%
where {\textless}number{\textgreater} is the maximum number of Jobs with the current
This directive specifies the maximum number of Jobs with the current
Storage resource that can read concurrently.
}

Expand Down

0 comments on commit c03310b

Please sign in to comment.