From 1fd7bb7054e9948327b88418b29f860f7030226a Mon Sep 17 00:00:00 2001 From: Joerg Steffens Date: Mon, 15 Sep 2014 18:36:45 +0200 Subject: [PATCH] updated explanations, some bugfixes and cleanup --- manuals/en/main/autochangers.tex | 129 +++++++++++++++------------- manuals/en/main/bareos.sty | 13 ++- manuals/en/main/dirdconf.tex | 118 ++++++++++++-------------- manuals/en/main/programs.tex | 1 + manuals/en/main/storedconf.tex | 140 +++++++++++++++++-------------- 5 files changed, 211 insertions(+), 190 deletions(-) diff --git a/manuals/en/main/autochangers.tex b/manuals/en/main/autochangers.tex index f4e3f6e..859aba5 100644 --- a/manuals/en/main/autochangers.tex +++ b/manuals/en/main/autochangers.tex @@ -853,27 +853,30 @@ \section{Bareos Autochanger Interface} \section{Tapespeed and blocksizes} \index[general]{speedtuning!tapedrives} +\index[general]{tuning!tape} \index[general]{blocksize!tapedrives} -\index[general]{tapespeed and blocksizes} +\index[general]{tape speed} +\index[general]{block sizes} \label{Tapespeed and blocksizes} +\label{setblocksizes} -The tape speed tuning whitepaper ( \elink{http://www.bareos.org/en/Whitepapers/articles/Speed\_Tuning\_of\_Tape\_Drives.html}{http://www.bareos.org/en/Whitepapers/articles/Speed\_Tuning\_of\_Tape\_Drives.html}) +The tape speed tuning whitepaper (\elink{http://www.bareos.org/en/Whitepapers/articles/Speed\_Tuning\_of\_Tape\_Drives.html}{http://www.bareos.org/en/Whitepapers/articles/Speed\_Tuning\_of\_Tape\_Drives.html}) shows that the two parameters -{\bf maximum file size} and {\bf maximum block size} of the device +\configdirective{maximum file size} and \configdirective{maximum block size} of the device have siginificant influence on the tape speed. -While it is no problem to change the "maximum file size" parameter, -unfortunately it is not possible to change the "maximum block size" +While it is no problem to change the \configdirective{maximum file size} parameter, +unfortunately it is not possible to change the \configdirective{maximum block size} parameter, because the previously written tapes would become unreadable -in the new setup. It would require that the "maximum block size" parameter +in the new setup. It would require that the \configdirective{maximum block size} parameter is switched back to the old value to be able to read the old volumes, but of course then the new volumes would be unreadable. Why is that the case? -The problem is that bareos writes the bareos label block (header) in the same block -size that is configured in the {\bf maximum block size} parameter in the device. -Per default, this value is 63k, so usually a tape written by bareos looks like this: +The problem is that Bareos writes the label block (header) in the same block +size that is configured in the \configdirective{maximum block size} parameter in the device. +Per default, this value is 63k, so usually a tape written by Bareos looks like this: \begin{verbatim} |------------------ @@ -900,27 +903,26 @@ \section{Tapespeed and blocksizes} As you can see, every block is written with the maximum block size, also the label block. The problem that arises here is that reading a block header with a wrong block size causes -a read error which is interpreted as an non-existent label by bareos. +a read error which is interpreted as an non-existent label by Bareos. -This is a potential source of data loss, because in normal operation, bareos refuses to +This is a potential source of data loss, because in normal operation, Bareos refuses to relabel an already labeled volume to be sure to not overwrite data that is still needed. -If bareos cannot read the volume label, this security mechanism does not work and you might +If Bareos cannot read the volume label, this security mechanism does not work and you might label tapes already labeled accidentially. -To solve this problem, the block size handling in bareos was changed in the following way: +To solve this problem, the block size handling was changed in Bareos \sinceVersion{sd}{Maximum Block Size}{14.2} in the following way: \begin{itemize} \item The tape label block is always written in the standard 63k (64512) Blocksize. \item The following blocks are then written in the block size configured in the {\bf maximum block size} directive. \item To be able to change the block size in an existing environment, it is now -possible to set the "maximum block size" and "minimum block size" in the +possible to set the \configdirective{maximum block size} and \configdirective{minimum block size} in the pool ressource. This setting is automatically promoted to each medium in -that pool as usual (i.e. when a medium is labelled for that pool or if a volume is transferred to that pool from the scratch pool). +that pool as usual (i.e. when a medium is labeled for that pool or if a volume is transferred to that pool from the scratch pool). When a volume is mounted, the volume's block size is used to write and read the data blocks that follow the header block. \end{itemize} -The following picture shows the result: We have a label block with a certain size (63k per default to be compatible to old installations), -and the following data blocks are written with another blocksize. +The following picture shows the result: \begin{verbatim} |--------------------------------| |label block (label block size) | @@ -931,6 +933,8 @@ \section{Tapespeed and blocksizes} |data block n(maximum block size)| ---------------------------------| \end{verbatim} +We have a label block with a certain size (63k per default to be compatible to old installations), +and the following data blocks are written with another blocksize. This approach has the following advantages: @@ -938,12 +942,12 @@ \section{Tapespeed and blocksizes} \item If nothing is configured, existing installations keep on working without problems. \item If you want to switch an existing installation that uses the default blocksize and move to a new (usually bigger) block size, you can do that -easily by creating a new pool, where {\bf maximum block size} is set to the new +easily by creating a new pool, where \configdirective{maximum block size} is set to the new value that you wish to use in the future: \end{itemize} -\begin{verbatim} +\begin{bconfig}{Pool Ressource: setting Maximum Block Size} Pool { Name = LTO-4-1M Pool Type = Backup @@ -954,17 +958,17 @@ \section{Tapespeed and blocksizes} Recycle Pool = Scratch } -\end{verbatim} +\end{bconfig} Now configure your backups that they will write into the newly defined pool in the future, and your backups will be written with the new blocksize. Your existing tapes can be automatically transferred to the new pool when they expire -via the {\bf scratch pool} mechanism. When a tape in your old pool expires, it is -transferred to the Scratch pool if you set +via the \ilink{Scratch Pool}{TheScratchPool} mechanism. When a tape in your old pool expires, it is +transferred to the scratch pool if you set {\bf Recycle Pool = Scratch}. When your new pool needs a new volume, it will get it from the scratch pool and apply the new pool's properties to that tape which also -include {\bf maximum block size} and {\bf minimum block size}. +include \configdirective{maximum block size} and \configdirective{minimum block size}. This way you can smoothly switch your tapes to a new block size while you can still restore the data on your old tapes at any time. @@ -974,35 +978,41 @@ \subsection{Possible Problems} is if you have used bigger block sizes already in your setup. As we now defined the label block to always be 63k, all labels will not be readable. -To also solve this problem, the directive {\bf label block size} can be used to define +To also solve this problem, the directive \configdirective{label block size} can be used to define a different label block size. That way, everything should work smoothly as all label blocks will be readable again. \subsection{How can I find out which blocksize was used when the tape was written?} -At least on linux, you can see if bareos tries to read the blocks with the -wrong block size. In that case, you get a message like the following in +At least on Linux, you can see if Bareos tries to read the blocks with the +wrong block size. In that case, you get a kernel message like the following in your system's messages: \begin{verbatim} [542132.410170] st1: Failed to read 1048576 byte block with 64512 byte transfer. \end{verbatim} Here, the block was written with 1M block size but we only read 64k. -\subsection{use of bls/bextract etc. with variable blocksizes} +\subsection{Direct access to Volumes with with non-default blocksizes} +\label{direct-access-to-volumes-with-non-default-blocksizes} +\index[general]{bls!block size} +\index[general]{bextract!block size} +\index[general]{Command!bls!block size} +\index[general]{Command!bextract!block size} -{\bf bls} and {\bf bextract} can directly access bareos volumes without catalog database. -This means that the {\bf maximum block size} information about the volume is also missing. + +\command{bls} and \command{bextract} can directly access Bareos volumes without catalog database. +This means that these programs don't have information about the used block size. To be able to read a volume written with an arbitrary blocksize, you need to -set the {\bf label block size}(to be able to to read the label block) and the -{\bf maximum block size} (to be able to read the data blocks) setting in the +set the \configdirective{label block size} (to be able to to read the label block) and the +\configdirective{maximum block size} (to be able to read the data blocks) setting in the device definition used by those tools to be able to open the medium. -Example using bls with a tape that was written with another blocksize than the -DEFAULT\_BLOCK\_SIZE (63k), but with the default label block size of 63k: -\begin{verbatim} -../sbin/bls FC-Drive-1 -V A00007L4 +Example using \command{bls} with a tape that was written with another blocksize than the +\variable{DEFAULT_BLOCK_SIZE} (63k), but with the default label block size of 63k: +\begin{commands}{bls with non-default block size} +bls FC-Drive-1 -V A00007L4 bls: butil.c:289-0 Using device: "FC-Drive-1" for reading. 25-Feb 12:47 bls JobId 0: No slot defined in catalog (slot=0) for Volume "A00007L4" on "FC-Drive-1" (/dev/tape/by-id/scsi-350011d00018a5f03-nst). 25-Feb 12:47 bls JobId 0: Cartridge change or "update slots" may be required. @@ -1011,53 +1021,54 @@ \subsection{use of bls/bextract etc. with variable blocksizes} Bareos status: file=0 block=1 Device status: ONLINE IM_REP_EN file=0 block=2 0 files found. -\end{verbatim} - -As can be seen, bls manages to read the label block as it knows what volume is mounted -(Ready to read from volume ''A00007L4''), but fails to read the data blocks. +\end{commands} -{\bf dmesg} shows the following: +As can be seen, \command{bls} manages to read the label block as it knows what volume is mounted +(Ready to read from volume \parameter{A00007L4}), but fails to read the data blocks. -\begin{verbatim} +\begin{commands}{dmesg} +dmesg +[...] st2: Failed to read 131072 byte block with 64512 byte transfer. -\end{verbatim} +[...] +\end{commands} This shows that the blocksize for the data blocks that we need is 131072. -Now we have to set this blocksize in the bareos-sd.conf, Device -resource as "Maximum Block Size": +Now we have to set this blocksize in the \file{bareos-sd.conf}, device +resource as \configdirective{Maximum Block Size}: -\begin{verbatim} +\begin{bconfig}{Storage Device Ressource: setting Maximum Block Size} Device { - Name = FC-Drive-1 # + Name = FC-Drive-1 Drive Index = 0 Media Type = LTO-4 Archive Device = /dev/tape/by-id/scsi-350011d00018a5f03-nst - AutomaticMount = yes; # when device opened, read it - AlwaysOpen = yes; - RemovableMedia = yes; - RandomAccess = no; - AutoChanger = yes; + AutomaticMount = yes + AlwaysOpen = yes + RemovableMedia = yes + RandomAccess = no + AutoChanger = yes Maximum Block Size = 131072 } - -\end{verbatim} +\end{bconfig} Now we can call bls again, and everything works as expected: -\begin{verbatim} -sbin/bls FC-Drive-1 -V A00007L4 +\begin{commands}{bls with non-default block size} +bls FC-Drive-1 -V A00007L4 bls: butil.c:289-0 Using device: "FC-Drive-1" for reading. 25-Feb 12:49 bls JobId 0: No slot defined in catalog (slot=0) for Volume "A00007L4" on "FC-Drive-1" (/dev/tape/by-id/scsi-350011d00018a5f03-nst). 25-Feb 12:49 bls JobId 0: Cartridge change or "update slots" may be required. 25-Feb 12:49 bls JobId 0: Ready to read from volume "A00007L4" on device "FC-Drive-1" (/dev/tape/by-id/scsi-350011d00018a5f03-nst). -bls JobId 203: -rw-r--r-- 1 pstorz users 221 2014-02-25 09:52:51 /home/pstorz/.java/.userPrefs/_!':!bw"t!#4!}@"w!(@!}@"u!'%!.g"k!')!.g"p!'@!~@!t!'k!~!==/prefs.xml -bls JobId 203: drwxr-xr-x 2 pstorz users 4096 2014-02-25 09:52:51 /home/pstorz/.java/.userPrefs/_!':!bw"t!#4!}@"w!(@!}@"u!'%!.g"k!')!.g"p!'@!~@!t!'k!~!==/ -\end{verbatim} +bls JobId 203: [...] +\end{commands} \subsection{How to configure the blocksizes in your environment} The following chart shows how to set the directives for {\bf maximum block size} and {\bf label block size} depending on how your current setup is: -\includegraphics[scale=0.8]{\idir blocksize-decisionchart} +\begin{center} +\includegraphics[width=0.8\linewidth]{\idir blocksize-decisionchart} +\end{center} diff --git a/manuals/en/main/bareos.sty b/manuals/en/main/bareos.sty index 6fbb7f6..755086d 100644 --- a/manuals/en/main/bareos.sty +++ b/manuals/en/main/bareos.sty @@ -129,7 +129,7 @@ % 3: directive type/value % 4: "required" | "" % 5: default value -% 6: since version +% 6: "deprecated" | since version % 7: description \item [#2 = {\textless}#3{\textgreater}]% \hfill% @@ -138,10 +138,15 @@ \hfill \\% \index[#1]{#2}% \index[#1]{Directive!#2}% + \ifthenelse{\equal{#6}{deprecated}}{% +\warning{This directive is deprecated.\\} + }{}% #7\ \\% - \ifthenelse{\isempty{#6}}{}{% - \sinceVersion{#1}{#2}{#6}% - %\marginpar{#6} + \ifthenelse{\equal{#6}{deprecated}}{}{% + \ifthenelse{\isempty{#6}}{}{% + \sinceVersion{#1}{#2}{#6}% + %\marginpar{#6} + }% }% } % \xdirective{dir|sd|fd}{name}{type}{required}{default}{version}{} diff --git a/manuals/en/main/dirdconf.tex b/manuals/en/main/dirdconf.tex index 570e02a..a966e83 100644 --- a/manuals/en/main/dirdconf.tex +++ b/manuals/en/main/dirdconf.tex @@ -328,7 +328,7 @@ \section{Director Resource} See chapter about \ilink{how to extract information from the catalog}{UseBareosCatalogToExtractInformationChapter} for additional information. -See the \ilink{ Configuration chapter}{Time} of this manual for additional +See the \ilink{Configuration chapter}{Time} of this manual for additional details of time specification. The default is 5 years. @@ -2786,26 +2786,13 @@ \section{Pool Resource} Start of the Pool resource. There must be at least one Pool resource defined. -\item [Name = {\textless}name{\textgreater}] \hfill \\ -\index[dir]{Name} -\index[dir]{Directive!Name} -The name of the pool. For most applications, you will use the default -pool name {\bf Default}. This directive is required. -\label{MaxVolumes} -\item [Maximum Volumes = {\textless}number{\textgreater}] \hfill \\ -\index[dir]{Maximum Volumes} -\index[dir]{Directive!Maximum Volumes} -This directive specifies the maximum number of volumes (tapes or files) -contained in the pool. This directive is optional, if omitted or set to -zero, any number of volumes will be permitted. In general, this -directive is useful for Autochangers where there is a fixed number of -Volumes, or for File storage where you wish to ensure that the backups -made to disk files do not become too numerous or consume too much space. +\xdirective{dir}{Name}{name}{required}{}{}{% +The name of the pool. +} + -\item [Pool Type = {\textless}type{\textgreater}] \hfill \\ -\index[dir]{Pool Type} -\index[dir]{Directive!Pool Type} +\xdirective{dir}{Pool Type}{backup \textbar \ldots}{}{}{}{% This directive defines the pool type, which corresponds to the type of Job being run. It is required and may be one of the following: @@ -2818,41 +2805,45 @@ \section{Pool Resource} \item [*Save] \end{description} Note, only Backup is currently implemented. +} -\item [Storage = {\textless}storage-resource-name{\textgreater}] \hfill \\ -\index[dir]{Storage} -\index[dir]{Directive!Storage} +\xdirective{dir}{Storage}{storage-resource-name}{}{}{}{% The Storage directive defines the name of the storage services where you want to backup the FileSet data. For additional details, see the -\ilink{Storage Resource Chapter}{StorageResource2} of this manual. +\ilink{Storage Resource chapter}{StorageResource2} of this manual. The Storage resource may also be specified in the Job resource, but the value, if any, in the Pool resource overrides any value in the Job. This Storage resource definition is not required by either the Job resource or in the Pool, but it must be specified in one or the other. If not configuration error will result. +} -\item [Use Volume Once = {\textless}yes{\textbar}no{\textgreater}] \hfill \\ -\index[dir]{Use Volume Once} -\index[dir]{Directive!Use Volume Once} -This directive if set to {\bf yes} specifies that each volume is to be -used only once. This is most useful when the Media is a file and you -want a new file for each backup that is done. The default is {\bf no} -(i.e. use volume any number of times). This directive will most likely -be phased out (deprecated), so you are recommended to use {\bf Maximum -Volume Jobs = 1} instead. -The value defined by this directive in the bareos-dir.conf file is the -default value used when a Volume is created. Once the volume is -created, changing the value in the bareos-dir.conf file will not change -what is stored for the Volume. To change the value for an existing -Volume you must use the {\bf update} command in the Console. +\xdirective{dir}{Minimum Block Size}{size-in-bytes}{}{}{14.2}{% +The \configdirective{Minimum Block Size} can be defined here or at \ilink{Storage Device resource}{storage-device-minimumblocksize}. +For details, see chapter \ilink{Setting Block Sizes}{setblocksizes}. +} + + +\xdirective{dir}{Maximum Block Size}{size-in-bytes}{}{}{14.2}{% +The \configdirective{Maximum Block Size} can be defined here or at \ilink{Storage Device resource}{storage-device-maximumblocksize}. +If not defined, its default is 63 KB. +Increasing this value could improve the throughput of writing to tapes a lot, see \ilink{Setting Block Sizes}{setblocksizes} chapter. +} + + +\xdirective{dir}{Maximum Volumes}{number}{}{0 (no limit)}{}{% +\label{MaxVolumes}% +This directive specifies the maximum number of volumes (tapes or files) +contained in the pool. This directive is optional, if omitted or set to +zero, any number of volumes will be permitted. In general, this +directive is useful for Autochangers where there is a fixed number of +Volumes, or for File storage where you wish to ensure that the backups +made to disk files do not become too numerous or consume too much space. +} -Please see the notes below under {\bf Maximum Volume Jobs} concerning -using this directive with multiple simultaneous jobs. -\item [Maximum Volume Jobs = {\textless}positive-integer{\textgreater}] \hfill \\ -\index[dir]{Maximum Volume Jobs} -\index[dir]{Directive!Maximum Volume Jobs} +\xdirective{dir}{Maximum Volume Jobs}{number}{}{0 (no limit)}{}{% This directive specifies the maximum number of Jobs that can be written to the Volume. If you specify zero (the default), there is no limit. Otherwise, when the number of Jobs backed up to the Volume equals {\bf @@ -2874,10 +2865,9 @@ \section{Pool Resource} start writing to the Volume. At some point, when the Media record is updated, multiple simultaneous jobs may fail since the Volume can no longer be written. +} -\item [Maximum Volume Files = {\textless}positive-integer{\textgreater}] \hfill \\ -\index[dir]{Maximum Volume Files} -\index[dir]{Directive!Maximum Volume Files} +\xdirective{dir}{Maximum Volume Files}{number}{}{0 (no limit)}{}{% This directive specifies the maximum number of files that can be written to the Volume. If you specify zero (the default), there is no limit. Otherwise, when the number of files written to the Volume equals {\bf @@ -2893,10 +2883,9 @@ \section{Pool Resource} created, changing the value in the bareos-dir.conf file will not change what is stored for the Volume. To change the value for an existing Volume you must use the {\bf update} command in the Console. +} -\item [Maximum Volume Bytes = {\textless}size{\textgreater}] \hfill \\ -\index[dir]{Maximum Volume Bytes} -\index[dir]{Directive!Maximum Volume Bytes} +\xdirective{dir}{Maximum Volume Bytes}{number}{}{0 (no limit)}{}{% This directive specifies the maximum number of bytes that can be written to the Volume. If you specify zero (the default), there is no limit except the physical size of the Volume. Otherwise, when the number of @@ -2916,6 +2905,7 @@ \section{Pool Resource} created, changing the value in the bareos-dir.conf file will not change what is stored for the Volume. To change the value for an existing Volume you must use the {\bf update} command in the Console. +} \xdirective{dir}{Volume Use Duration}{time-period-specification}{}{0 (no limit)}{} {The Volume Use Duration directive defines the time period that the @@ -2981,23 +2971,20 @@ \section{Pool Resource} the client (and the Catalog definition in a Job since 13.4.0). e.g. this catalog setting takes precedence over any other definition. -\label{PoolAutoPrune} -\item [AutoPrune = {\textless}yes{\textbar}no{\textgreater}] \hfill \\ -\index[dir]{AutoPrune} -\index[dir]{Directive!AutoPrune} -If AutoPrune is set to {\bf yes} (default), Bareos (version 1.20 or -greater) will automatically apply the Volume Retention period when new +\xdirective{dir}{Auto Prune}{yes{\textbar}no}{}{yes}{}{% +\label{PoolAutoPrune}% +If AutoPrune is set to {\bf yes}, Bareos will automatically +apply the Volume Retention period when new Volume is needed and no appendable Volumes exist in the Pool. Volume pruning causes expired Jobs (older than the {\bf Volume Retention} period) to be deleted from the Catalog and permits possible recycling of the Volume. +} +\xdirective{dir}{Volume Retention}{time-period-specification}{}{365 days}{}{% \label{VolRetention} -\item [Volume Retention = {\textless}time-period-specification{\textgreater}] \hfill \\ -\index[dir]{Volume Retention} -\index[dir]{Directive!Volume Retention} -The Volume Retention directive defines the length of time that {\bf -Bareos} will keep records associated with the Volume in +The Volume Retention directive defines the length of time that +Bareos will keep records associated with the Volume in the Catalog database after the End time of each Job written to the Volume. When this time period expires, and if {\bf AutoPrune} is set to {\bf yes} Bareos may prune (remove) Job records that are older than the @@ -3035,9 +3022,10 @@ \section{Pool Resource} The default Volume retention period is 365 days, and either the default or the value defined by this directive in the bareos-dir.conf file is the default value used when a Volume is created. Once the volume is -created, changing the value in the bareos-dir.conf file will not change +created, changing the value in the \file{bareos-dir.conf} file will not change what is stored for the Volume. To change the value for an existing Volume you must use the {\bf update} command in the Console. +} \item [Action On Purge = {\textless}Truncate{\textgreater}] \hfill \\ \index[dir]{actiononpurge} @@ -3297,9 +3285,13 @@ \section{Pool Resource} of this manual. In almost all cases, you should enclose the format specification (part -after the equal sign) in double quotes. Please note that this directive -is deprecated and is replaced in version 1.37 and greater with a Python -script for creating volume names. +after the equal sign) in double quotes. + + +\xdirective{dir}{Use Volume Once}{yes{\textbar}no}{}{no}{deprecated}{% +Use \configdirective{Maximum Volume Jobs = 1} instead. +} + \end{description} In order for a Pool to be used during a Backup Job, the Pool must have at diff --git a/manuals/en/main/programs.tex b/manuals/en/main/programs.tex index 94269aa..b4f6892 100644 --- a/manuals/en/main/programs.tex +++ b/manuals/en/main/programs.tex @@ -207,6 +207,7 @@ \section{Volume Utility Commands} \index[general]{Volume Utility Tools} \index[general]{Tools!Volume Utility} +\warning{If you using Bareos with non-default block sizes defined in the pools, it might be neccessary to specifiy the \configdirective{maximum block level} also in the storage device ressource, see \ilink{Direct access to Volumes with non-default blocksizes}{direct-access-to-volumes-with-non-default-blocksizes}.} \subsection{bls} \label{bls} diff --git a/manuals/en/main/storedconf.tex b/manuals/en/main/storedconf.tex index 242349b..a3e763d 100644 --- a/manuals/en/main/storedconf.tex +++ b/manuals/en/main/storedconf.tex @@ -54,38 +54,41 @@ \section{Storage Resource} \begin{description} -\directive{sd}{Name}{name}{required}{}{} +\xdirective{sd}{Name}{name}{required}{}{}{% Specifies the Name of the Storage daemon. +} -\directive{sd}{Working Directory}{directory}{required}{{\it platform specific}}{} +\xdirective{sd}{Working Directory}{directory}{required}{{\it platform specific}}{}{% This directive specifies a directory in which the Storage daemon may put its status files. This directory should be used only by {\bf Bareos}, but may be shared by other Bareos daemons provided the names given to each daemon are unique. +} -\directive{sd}{Pid Directory}{directory}{required}{{\it platform specific}}{} +\xdirective{sd}{Pid Directory}{directory}{required}{{\it platform specific}}{}{% This directive specifies a directory in which the Storage Daemon may put its process Id file files. The process Id file is used to shutdown Bareos and to prevent multiple copies of Bareos from running simultaneously. Standard shell expansion of the {\bf directory} is done when the configuration file is read so that values such as {\bf \$HOME} will be properly expanded. +} % \item [Subsys Directory = {\textless}Directory{\textgreater}] \hfill \\ % \index[sd]{Subsys Directory} % \index[sd]{Directive!Subsys Directory} % This directive is currently unused. -\directive{sd}{Plugin Directory}{directory}{required}{{\it platform specific}}{} +\xdirective{sd}{Plugin Directory}{directory}{required}{{\it platform specific}}{}{% This directive specifies a directory in which the Storage Daemon searches for plugins with the name {\bf {\textless}pluginname{\textgreater}-sd.so} which it will load at startup. +} %\directive{sd}{Scripts Directory}{directory}{required}{{\it platform specific}}{} %This directive is currently unused. -\directive{sd}{Heartbeat Interval}{time-interval}{required}{0}{} -\index[general]{Heartbeat Interval} -\index[general]{Broken pipe} +\xdirective{sd}{Heartbeat Interval}{time-interval}{required}{0}{}{% +\index[general]{Broken pipe}% This directive defines an interval of time in seconds. When the Storage daemon is waiting for the operator to mount a tape, each time interval, it will send a heartbeat signal to @@ -95,16 +98,17 @@ \section{Storage Resource} standards and times out an valid connection after a short duration despite the fact that keepalive is set. This usually results in a broken pipe error message. +} -\directive{sd}{Client Connect Wait}{time-interval}{required}{30 minutes}{} -\index[general]{Client Connect Wait} +\xdirective{sd}{Client Connect Wait}{time-interval}{required}{30 minutes}{}{% This directive defines an interval of time in seconds that the Storage daemon will wait for a Client (the File daemon) to connect. Be aware that the longer the Storage daemon waits for a Client, the more resources will be tied up. +} -\directive{sd}{Maximum Concurrent Jobs}{number}{required}{10}{} +\xdirective{sd}{Maximum Concurrent Jobs}{number}{required}{10}{}{% where {\textless}number{\textgreater} is the maximum number of Jobs that may run concurrently. The default is set to 10, but you may set it to a larger number. Each contact from the Director (e.g. status request, job start @@ -116,23 +120,25 @@ \section{Storage Resource} will almost certainly need to set the {\bf Maximum Concurrent Jobs} in the Storage resource in the Director's configuration file and possibly those in the Job and Client resources. +} -\directive{sd}{SD Adress}{IP-address}{}{}{} +\xdirective{sd}{SD Address}{IP-address}{}{}{}{% This directive is optional, and if it is specified, it will cause the Storage daemon server (for Director and File daemon connections) to bind to the specified {\bf IP-Address}, which is either a domain name or an IP address specified as a dotted quadruple. If this directive is not specified, the Storage daemon will bind to any available address (the default). +} -\directive{sd}{SD Adresses}{IP-address-specification}{}{}{} +% failed to use xdirective here, because of verbatim including {} +\directive{sd}{SD Addresses}{IP-address-specification}{}{}{}% Specify the ports and addresses on which the Storage daemon will listen for Director connections. Normally, the default is sufficient and you do not need to specify this directive. Probably the simplest way to explain how this directive works is to show an example: -\footnotesize -\begin{verbatim} +\begin{bconfig}{SD Addresses} SDAddresses = { ip = { addr = 1.2.3.4; port = 1205; } ipv4 = { @@ -155,8 +161,7 @@ \section{Storage Resource} addr = bluedot.thun.net } } -\end{verbatim} -\normalsize +\end{bconfig} where ip, ip4, ip6, addr, and port are all keywords. Note, that the address can be specified as either a dotted quadruple, or IPv6 colon notation, or as @@ -169,11 +174,13 @@ \section{Storage Resource} Using this directive, you can replace both the SDPort and SDAddress directives shown below. -\directive{sd}{SD Port}{port-number}{required}{9103}{} + +\xdirective{sd}{SD Port}{port-number}{required}{9103}{}{% Specifies port number on which the Storage daemon listens for Director connections. +} -\directive{sd}{Compatible}{yes{\textbar}no}{required}{yes}{} +\xdirective{sd}{Compatible}{yes{\textbar}no}{required}{yes}{}{% This directive enables the compatible mode of the storage daemon. In this mode the storage daemon will try to write the storage data in a compatible way with Bareos of which Bareos is a fork. This only works @@ -181,33 +188,40 @@ \section{Storage Resource} are Bareos specific. Which may be read when used by a Bareos storage daemon but might not be understood by any of the Bareos components (dir/sd/fd). +} -\directive{sd}{NDMP Enable}{yes{\textbar}no}{}{}{} +\xdirective{sd}{NDMP Enable}{yes{\textbar}no}{}{}{}{% This directive enables the Native NDMP Tape Agent. +} -\directive{sd}{NDMP Snooping}{yes{\textbar}no}{}{}{} +\xdirective{sd}{NDMP Snooping}{yes{\textbar}no}{}{}{}{% This directive enables the Snooping and pretty printing of NDMP protocol information in debugging mode. +} -\directive{sd}{NDMP Loglevel}{level}{}{}{} +\xdirective{sd}{NDMP Loglevel}{level}{}{}{}{% This directive sets the loglevel for the NDMP protocol library. +} -\directive{sd}{NDMP Address}{IP-address}{}{}{} +\xdirective{sd}{NDMP Address}{IP-address}{}{}{}{% This directive is optional, and if it is specified, it will cause the Storage daemon server (for NDMP Tape Server connections) to bind to the specified {\bf IP-Address}, which is either a domain name or an IP address specified as a dotted quadruple. If this directive is not specified, the Storage daemon will bind to any available address (the default). +} -\directive{sd}{NDMP Addresses}{IP-address-specification}{}{}{} +\xdirective{sd}{NDMP Addresses}{IP-address-specification}{}{}{}{% Specify the ports and addresses on which the Storage daemon will listen for NDMP Tape Server connections. Normally, the default is sufficient and you do not need to specify this directive. +} -\directive{sd}{NDMP Port}{port-specification}{}{10000}{} +\xdirective{sd}{NDMP Port}{port-specification}{}{10000}{}{% Specifies port number on which the Storage daemon listens for NDMP Tape Server connections. +} \xdirective{sd}{AutoXflateOnReplication}{yes{\textbar}no}{required}{yes}{13.4}{This directive controls the autoxflate-sd plugin when replicating data inside one or between two storage daemons (Migration/Copy Jobs). Normally the storage daemon will @@ -222,7 +236,7 @@ \section{Storage Resource} The following is a typical Storage daemon Storage definition. -\begin{bconfig}{Storage daemon Storage definition} +\begin{bconfig}{Storage daemon storage definition} # # "Global" Storage daemon configuration specifications appear # under the Storage resource. @@ -371,12 +385,12 @@ \section{Device Resource} There are three different types that are supported: \begin{description} \item[device] Usually the device file -name of a removable storage device (tape drive), for example "{\bf -/dev/nst0}" or "{\bf /dev/rmt/0mbn}", preferably in the "non-rewind" variant. +name of a removable storage device (tape drive), for example \path|/dev/nst0| +or \path|/dev/rmt/0mbn|, preferably in the "non-rewind" variant. In addition, on systems such as Sun, which have multiple tape access methods, you must be sure to specify to use Berkeley I/O conventions with the device. The {\bf b} in the Solaris (Sun) archive -specification {\bf /dev/rmt/0mbn} is what is needed in this case. +specification \path|/dev/rmt/0mbn| is what is needed in this case. Bareos does not support SysV tape drive behavior. \item[directory] If a directory is specified, it is used as file storage. The directory must be existing and be specified as absolute path. @@ -386,7 +400,7 @@ \section{Device Resource} the load to different disk drives), you will need to define two Device resources, each containing an Archive Device with a different directory. \item[fifo] \label{SetupFifo} -A FIFO is a special kind of file that connects two programs +A FIFO is a special kind of file that connects two programs via kernel memory. If a FIFO device is specified for a backup operation, you must have a program that reads what Bareos writes into the FIFO. When the Storage daemon starts the job, it will wait for {\bf MaximumOpenWait} seconds @@ -399,9 +413,8 @@ \section{Device Resource} to read a label of a FIFO device, but will simply write on it. To create a FIFO Volume in the catalog, use the {\bf add} command rather than the {\bf label} command to avoid attempting to write a label. -{ -\footnotesize -\begin{verbatim} + +\begin{bconfig}{Fifo device} Device { Name = FifoStorage Media Type = Fifo @@ -414,9 +427,8 @@ \section{Device Resource} MaximumOpenWait = 60 AlwaysOpen = no } -\end{verbatim} -} -%\normalsize +\end{bconfig} + During a restore operation, if the Archive Device is a FIFO, Bareos will attempt to read from the FIFO, so you must have an external program that writes into the FIFO. Bareos will wait {\bf MaximumOpenWait} seconds for the @@ -669,28 +681,30 @@ \section{Device Resource} {\bf Always Open} set to {\bf Yes} -\item [Volume Poll Interval = {\textless}time{\textgreater}] \hfill \\ -\index[sd]{Volume Poll Interval} -\index[sd]{Directive!Volume Poll Interval} -If the time specified on this directive is non-zero, after asking the -operator to mount a new volume Bareos will periodically poll (or read) the +\xdirective{sd}{Volume Poll Interval}{time-interval}{}{300}{}{% +If the time specified on this directive is non-zero, +%after asking the operator to mount a new volume +Bareos will periodically poll (or read) the drive at the specified interval to see if a new volume has been mounted. If -the time interval is zero (the default), no polling will occur. This +the time interval is zero, no polling will occur. This directive can be useful if you want to avoid operator intervention via the console. Instead, the operator can simply remove the old volume and insert the requested one, and Bareos on the next poll will recognize the new tape -and continue. Please be aware that if you set this interval too small, you +and continue. +Please be aware that if you set this interval too small, you may excessively wear your tape drive if the old tape remains in the drive, -since Bareos will read it on each poll. This can be avoided by ejecting the -tape using the {\bf Offline On Unmount} and the {\bf Close on Poll} -directives. -However, if you are using a Linux 2.6 kernel or other OSes -such as FreeBSD or Solaris, the Offline On Unmount will leave the drive -with no tape, and Bareos will not be able to properly open the drive and -may fail the job. +since Bareos will read it on each poll. +% This can be avoided by ejecting the +% tape using the {\bf Offline On Unmount} and the {\bf Close on Poll} +% directives. +% However, if you are using a Linux 2.6 kernel or other OSes +% such as FreeBSD or Solaris, the Offline On Unmount will leave the drive +% with no tape, and Bareos will not be able to properly open the drive and +% may fail the job. %\TODO{reference is missing: For more information on this problem, please see the %\ilink{description of Offline On Unmount}{NoTapeInDrive} in the Tape %Testing chapter.} +} \item [Close on Poll= {\textless}yes{\textbar}no{\textgreater}] \hfill \\ \index[sd]{Close on Poll} @@ -836,7 +850,8 @@ \section{Device Resource} drives or for disk Volumes where doing so may allow corrupted data to go undetected. -\directive{sd}{Minimum block size}{size-in-bytes}{}{0}{} +% xdirective? too long? +\directive{sd}{Minimum block size}{size-in-bytes}{}{0}{}% This statement applies only to non-random access devices (e.g. tape drives). Blocks written by the storage daemon to a non-random archive device will never be smaller than the given {\bf size-in-bytes}. @@ -864,16 +879,18 @@ \section{Device Resource} identical to the size specified in Bareos -- otherwise when you attempt to re-read your Volumes, you will get an error. -If you want the block size to be variable but with a 64K minimum and 200K +If you want the block size to be variable but with a 63K minimum and 200K maximum (and default as well), you would specify: \begin{bconfig}{Variable Block Size} -Minimum block size = 64K +Minimum block size = 63K Maximum block size = 200K \end{bconfig} -\directive{sd}{Maximum block size}{size-in-bytes}{}{64512}{} + +\xdirective{sd}{Maximum block size}{size-in-bytes}{}{64512}{}{ +\label{storage-device-maximumblocksize}% The Storage daemon will always attempt to write blocks of the specified {\bf size-in-bytes} to the archive device. As a consequence, this statement specifies both the default block size @@ -885,23 +902,18 @@ \section{Device Resource} If no value is specified or zero is specified, the Storage daemon will use a default block size of 64,512 bytes (126 * 512). -This default size is far too low for modern tape drives. If you -are using a modern tape drive like LTO-4 or newer, you should use -a block size of 512k or 1M. - -For more information, please see the tape speed whitepaper: -\elink{http://www.bareos.org/en/Whitepapers/articles/Speed\_Tuning\_of\_Tape\_Drives.html}{http://www.bareos.org/en/Whitepapers/articles/Speed\_Tuning\_of\_Tape\_Drives.html} - -%The maximum {\bf size-in-bytes} possible is 2,000,000. +\warning{If your are using LTO drives, changing the block size after labeling the tape will result into unreadable tapes.} -\warning{If your are using LTO drives, changing the block size after labeling the tape will result into unreadable tapes. So normally you will not change the block size in an existing configuration.} +Please read chapter \ilink{Tapespeed and blocksizes}{Tapespeed and blocksizes}, +to see how to tune this value in a safe manner. +} -\directive{sd}{Label block size}{size-in-bytes}{}{64512}{} +\xdirective{sd}{Label block size}{size-in-bytes}{}{64512}{14.2}{ The storage daemon will write the label blocks with the size configured here. Usually, you will not need to change this directive. For more information on this directive, please see \ilink{Tapespeed and blocksizes}{Tapespeed and blocksizes}. - +} \directive{sd}{Hardware End of Medium}{yes{\textbar}no}{}{yes}{} All modern (after 1998) tape drives should support this