This repository has been archived by the owner on Mar 19, 2021. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 16
/
dirdconf-fileset.tex
1771 lines (1489 loc) · 70.6 KB
/
dirdconf-fileset.tex
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
\subsection{FileSet Include Ressource}
\label{fileset-include}
The Include resource must contain a list of directories and/or files to be
processed in the backup job.
Normally, all files found in all
subdirectories of any directory in the Include File list will be backed up.
Note, see below for the definition of {\textless}file-list{\textgreater}.
The Include resource may also contain one or more Options resources that
specify options such as compression to be applied to all or any subset of
the files found when processing the file-list for backup. Please see
below for more details concerning Options resources.
There can be any number of {\bf Include} resources within the FileSet, each
having its own list of directories or files to be backed up and the backup
options defined by one or more Options resources.
Please take note of the following items in the FileSet syntax:
\begin{enumerate}
\item There is no equal sign (=) after the Include and before the opening
brace (\{). The same is true for the Exclude.
\item Each directory (or filename) to be included or excluded is preceded by a {\bf File
=}. Previously they were simply listed on separate lines.
\item The Exclude resource does not accept Options.
\item When using wild-cards or regular expressions, directory names are
always terminated with a slash (/) and filenames have no trailing slash.
\end{enumerate}
\begin{description}
\directive{dir}{File}{ filename \textbar\ dirname \textbar\ \textbar command \textbar\ \textbackslash\textless includefile-client \textbar\ \textless includefile-server }{}{}{}
The file list
consists of one file or directory name per line. Directory names should be
specified without a trailing slash with Unix path notation.
Windows users, please take note to specify directories (even c:/...) in
Unix path notation. If you use Windows conventions, you will most likely
not be able to restore your files due to the fact that the Windows
path separator was defined as an escape character long before Windows
existed, and Bareos adheres to that convention (i.e. means the next character
appears as itself).
You should always specify a full path for every directory and file that you
list in the FileSet. In addition, on Windows machines, you should {\bf
always} prefix the directory or filename with the drive specification
(e.g. {\bf c:/xxx}) using Unix directory name separators
(forward slash). The drive letter itself can be upper or lower case (e.g.
c:/xxx or C:/xxx).
Bareos's default for processing directories is to recursively descend in
the directory saving all files and subdirectories. Bareos will not by
default cross filesystems (or mount points in Unix parlance). This means
that if you specify the root partition (e.g. {\bf /}), Bareos will save
only the root partition and not any of the other mounted filesystems.
Similarly on Windows systems, you must explicitly specify each of the
drives you want saved (e.g.
{\bf c:/} and {\bf d:/} ...). In addition, at least for Windows systems, you
will most likely want to enclose each specification within double quotes
particularly if the directory (or file) name contains spaces. The {\bf df}
command on Unix systems will show you which mount points you must specify to
save everything. See below for an example.
Take special care not to include a directory twice or Bareos will backup
the same files two times wasting a lot of space on your archive device.
Including a directory twice is very easy to do. For example:
\begin{bconfig}{File Set}
Include {
Options {
compression=GZIP
}
File = /
File = /usr
}
\end{bconfig}
on a Unix system where /usr is a subdirectory (rather than a mounted
filesystem) will cause /usr to be backed up twice.
{\bf {\textless}file-list{\textgreater}} is a list of directory and/or filename names
specified with a {\bf File =} directive. To include names containing spaces,
enclose the name between double-quotes. Wild-cards are not interpreted
in file-lists. They can only be specified in Options resources.
There are a number of special cases when specifying directories and files in a
{\bf file-list}. They are:
\begin{itemize}
\item Any name preceded by an at-sign (@) is assumed to be the name of a
file, which contains a list of files each preceded by a "File =". The
named file is read once when the configuration file is parsed during the
Director startup. Note, that the file is read on the Director's machine
and not on the Client's. In fact, the @filename can appear anywhere
within the conf file where a token would be read, and the contents of
the named file will be logically inserted in the place of the @filename.
What must be in the file depends on the location the @filename is
specified in the conf file. For example:
\begin{bconfig}{File Set with Include File}
Include {
Options {
compression=GZIP
}
@/home/files/my-files
}
\end{bconfig}
\item Any name beginning with a vertical bar ({\textbar}) is assumed to
be the name of a program. This program will be executed on the Director's
machine at the time the Job starts (not when the Director reads the
configuration file), and any output from that program will be assumed to
be a list of files or directories, one per line, to be included. Before
submitting the specified command Bareos will performe
\ilink{character substitution}{character substitution}.
This allows you to have a job that, for example, includes all the local
partitions even if you change the partitioning by adding a disk. The
examples below show you how to do this. However, please note two
things: \\
1. if you want the local filesystems, you probably should be
using the new {\bf fstype} directive, which was added in version 1.36.3
and set {\bf onefs=no}.
\\
2. the exact syntax of the command needed in the examples below is very
system dependent. For example, on recent Linux systems, you may need to
add the -P option, on FreeBSD systems, the options will be different as
well.
In general, you will need to prefix your command or commands with a {\bf
sh -c} so that they are invoked by a shell. This will not be the case
if you are invoking a script as in the second example below. Also, you
must take care to escape (precede with a \textbackslash{}) wild-cards,
shell character, and to ensure that any spaces in your command are
escaped as well. If you use a single quotes (') within a double quote
("), Bareos will treat everything between the single quotes as one field
so it will not be necessary to escape the spaces. In general, getting
all the quotes and escapes correct is a real pain as you can see by the
next example. As a consequence, it is often easier to put everything in
a file and simply use the file name within Bareos. In that case the
{\bf sh -c} will not be necessary providing the first line of the file
is {\bf \#!/bin/sh}.
As an example:
\begin{bconfig}{File Set with inline script}
Include {
Options {
signature = SHA1
}
File = "|sh -c 'df -l | grep \"^/dev/hd[ab]\" | grep -v \".*/tmp\" | awk \"{print \\$6}\"'"
}
\end{bconfig}
% workaround for kile editor
\hide{$}
will produce a list of all the local partitions on a Linux system.
Quoting is a real problem because you must quote for Bareos which consists of
preceding every \textbackslash{} and every " with a \textbackslash{}, and
you must also quote for the shell command. In the end, it is probably easier
just to execute a script file with:
\begin{bconfig}{File Set with external script}
Include {
Options {
signature=MD5
}
File = "|my_partitions"
}
\end{bconfig}
where \command{my_partitions} has:
\footnotesize
\begin{verbatim}
#!/bin/sh
df -l | grep "^/dev/hd[ab]" | grep -v ".*/tmp" \
| awk "{print \$6}"
\end{verbatim}
\normalsize
If the vertical bar (\verb+|+) in front of \command{my_partitions} is preceded by a
backslash as in \textbackslash{}\verb+|+, the program will be executed on the
Client's machine instead of on the Director's machine.
Please note that if the filename is given within quotes, you
will need to use two slashes. An example, provided by John Donagher,
that backs up all the local UFS partitions on a remote system is:
\begin{bconfig}{File Set with inline script in quotes}
FileSet {
Name = "All local partitions"
Include {
Options {
signature=SHA1
onefs=yes
}
File = "\\|bash -c \"df -klF ufs | tail +2 | awk '{print \$6}'\""
}
}
\end{bconfig}
The above requires two backslash characters after the double quote (one
preserves the next one). If you are a Linux user, just change the {\bf ufs}
to {\bf ext3} (or your preferred filesystem type), and you will be in
business.
If you know what filesystems you have mounted on your system, e.g.
for Linux only using ext2, ext3 or ext4, you can backup
all local filesystems using something like:
\begin{bconfig}{File Set to backup all extfs partions}
Include {
Options {
signature = SHA1
onfs=no
fstype=ext2
}
File = /
}
\end{bconfig}
\item Any file-list item preceded by a less-than sign ({\textless}) will be taken
to be a file. This file will be read on the Director's machine (see
below for doing it on the Client machine) at the time
the Job starts, and the data will be assumed to be a list of directories or
files, one per line, to be included. The names should start in column 1 and
should not be quoted even if they contain spaces. This feature allows you to
modify the external file and change what will be saved without stopping and
restarting Bareos as would be necessary if using the @ modifier noted above.
For example:
\footnotesize
\begin{verbatim}
Include {
Options {
signature = SHA1
}
File = "</home/files/local-filelist"
}
\end{verbatim}
\normalsize
If you precede the less-than sign ({\textless}) with a backslash as in
\textbackslash{}{\textless}, the file-list will be read on the Client machine
instead of on the Director's machine. Please note that if the filename
is given within quotes, you will need to use two slashes.
\footnotesize
\begin{verbatim}
Include {
Options {
signature = SHA1
}
File = "\\</home/xxx/filelist-on-client"
}
\end{verbatim}
\normalsize
\item
\index[general]{Backing up!Partitions}
\index[general]{Backing up Raw Partitions}
If you explicitly specify a block device such as {\bf /dev/hda1}, then
Bareos will assume that this is a raw partition
to be backed up. In this case, you are strongly urged to specify a {\bf
sparse=yes} include option, otherwise, you will save the whole partition
rather than just the actual data that the partition contains. For example:
\begin{bconfig}{Backup Raw Partitions}
Include {
Options {
signature=MD5
sparse=yes
}
File = /dev/hd6
}
\end{bconfig}
will backup the data in device /dev/hd6. Note, the {bf /dev/hd6} must be
the raw partition itself. Bareos will not back it up as a raw device if
you specify a symbolic link to a raw device such as my be created by the
LVM Snapshot utilities.
\item A file-list may not contain wild-cards. Use directives in the
Options resource if you wish to specify wild-cards or regular expression
matching.
\end{itemize}
\directive{dir}{Exclude Dir Containing}{filename}{}{}{}
This directive can be added to the Include section of the FileSet resource. If the specified
filename ({\bf filename-string}) is found on the Client in any directory to be
backed up, the whole directory will be ignored (not backed up).
We recommend to use the filename \file{.nobackup}, as it is a hidden file on unix
systems, and explains what is the purpose of the file.
For example:
\begin{bconfig}{Exlude Directories containing the file .nobackup}
# List of files to be backed up
FileSet {
Name = "MyFileSet"
Include {
Options {
signature = MD5
}
File = /home
Exclude Dir Containing = .nobackup
}
}
\end{bconfig}
But in /home, there may be hundreds of directories of users and some
people want to indicate that they don't want to have certain
directories backed up. For example, with the above FileSet, if
the user or sysadmin creates a file named {\bf .nobackup} in
specific directories, such as
\begin{verbatim}
/home/user/www/cache/.nobackup
/home/user/temp/.nobackup
\end{verbatim}
then Bareos will not backup the two directories named:
\begin{verbatim}
/home/user/www/cache
/home/user/temp
\end{verbatim}
NOTE: subdirectories will not be backed up. That is, the directive
applies to the two directories in question and any children (be they
files, directories, etc).
\directive{dir}{Plugin}{plugin-name:plugin-parameter1:plugin-parameter2:$\ldots$}{}{}{}
\label{directive-fileset-plugin}
Instead of only specifying files, a file set can also use plugins.
Plugins are additional libraries that handle specific requirements.
The purpose of plugins is to provide an interface to any system program
for backup and restore. That allows you, for example, to do database backups without a local dump.
The syntax and semantics of the Plugin directive require
the first part of the string up to the colon to be the name of the plugin.
Everything after the first colon is ignored by the File daemon but is passed to the plugin.
Thus the plugin writer may define the
meaning of the rest of the string as he wishes.
The program \nameref{bpluginfo} can be used, to retreive information about a specific plugin.
Examples about the bpipe- and the mssql-plugin can be found in the sections about the \nameref{bpipe} and the \nameref{MSSQL}.
Note: It is also possible to define more than one plugin directive in a FileSet to do several database dumps at once.
\directive{dir}{Options}{$\ldots$}{}{}{}
See the \nameref{fileset-options} section.
\end{description}
\subsubsection{FileSet Options Ressource}
\label{fileset-options}
The Options resource is optional, but when specified, it will contain a
list of {\bf keyword=value} options to be applied to the file-list.
See below for the definition of file-list.
Multiple Options resources may be specified one after another. As the
files are found in the specified directories, the Options will applied to
the filenames to determine if and how the file should be backed up. The
wildcard and regular expression pattern matching parts of the
Options resources are checked in the order they are specified in the
FileSet until the first one that matches. Once one matches, the
compression and other flags within the Options specification will
apply to the pattern matched.
A key point is that in the absence of an Option or no other Option is
matched, every file is accepted for backing up. This means that if
you want to exclude something, you must explicitly specify an Option
with an {\bf exclude = yes} and some pattern matching.
Once Bareos determines that the Options resource matches the file under
consideration, that file will be saved without looking at any other Options
resources that may be present. This means that any wild cards must appear
before an Options resource without wild cards.
If for some reason, Bareos checks all the Options resources to a file under
consideration for backup, but there are no matches (generally because of wild
cards that don't match), Bareos as a default will then backup the file. This
is quite logical if you consider the case of no Options clause is specified,
where you want everything to be backed up, and it is important to keep in mind
when excluding as mentioned above.
However, one additional point is that in the case that no match was found,
Bareos will use the options found in the last Options resource. As a
consequence, if you want a particular set of "default" options, you should put
them in an Options resource after any other Options.
It is a good idea to put all your wild-card and regex expressions inside
double quotes to prevent conf file scanning problems.
This is perhaps a bit overwhelming, so there are a number of examples included
below to illustrate how this works.
You find yourself using a lot of Regex statements, which will cost quite a lot
of CPU time, we recommend you simplify them if you can, or better yet
convert them to Wild statements which are much more efficient.
%\input{autogenerated/datatype-options-table.tex}
The directives within an Options resource may be one of the following:
\begin{description}
\xdirective{Dir}{}{AutoExclude}{\dtYesNo}{}{yes}{14.2.2}{%
Automatically exclude files not intended for backup.
Currently only used for Windows, to exclude files defined in the registry key \registrykey{HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\BackupRestore\FilesNotToBackup}, see section \nameref{FilesNotToBackup}.
}
\item [compression={\textless}GZIP{\textbar}LZO{\textbar}LZFAST{\textbar}LZ4{\textbar}LZ4HC{\textgreater}] \hfill \\
\begin{description}
\item [compression=GZIP] \hfill \\
\index[dir]{compression}
\index[dir]{Directive!compression}
All files saved will be software compressed using the GNU ZIP
compression format. The compression is done on a file by file basis by
the File daemon. If there is a problem reading the tape in a single
record of a file, it will at most affect that file and none of the other
files on the tape. Normally this option is {\bf not} needed if you have
a modern tape drive as the drive will do its own compression. In fact,
if you specify software compression at the same time you have hardware
compression turned on, your files may actually take more space on the
volume.
Software compression is very important if you are writing your Volumes
to a file, and it can also be helpful if you have a fast computer but a
slow network, otherwise it is generally better to rely your tape drive's
hardware compression. As noted above, it is not generally a good idea
to do both software and hardware compression.
Specifying {\bf GZIP} uses the default compression level 6 (i.e. {\bf
GZIP} is identical to {\bf GZIP6}). If you want a different compression
level (1 through 9), you can specify it by appending the level number
with no intervening spaces to {\bf GZIP}. Thus {\bf compression=GZIP1}
would give minimum compression but the fastest algorithm, and {\bf
compression=GZIP9} would give the highest level of compression, but
requires more computation. According to the GZIP documentation,
compression levels greater than six generally give very little extra
compression and are rather CPU intensive.
You can overwrite this option per Storage resource with
\ilink{AllowCompression}{AllowCompression} option.
\item [compression=LZO] \hfill \\
\index[dir]{compression}
\index[dir]{Directive!compression}
All files saved will be software compressed using the LZO
compression format. The compression is done on a file by file basis by
the File daemon. Everything else about GZIP is true for LZO.
LZO provides much faster compression and decompression speed but lower
compression ratio than GZIP. If your CPU is fast enough you should be able
to compress your data without making the backup duration longer.
Note that Bareos only use one compression level LZO1X-1 specified by LZO.
You can overwrite this option per Storage resource with
\ilink{AllowCompression}{AllowCompression} option.
\item [compression=LZFAST] \hfill \\
\index[dir]{compression}
\index[dir]{Directive!compression}
All files saved will be software compressed using the LZFAST
compression format. The compression is done on a file by file basis by
the File daemon. Everything else about GZIP is true for LZFAST.
LZFAST provides much faster compression and decompression speed but lower
compression ratio than GZIP. If your CPU is fast enough you should be able
to compress your data without making the backup duration longer.
You can overwrite this option per Storage resource with
\ilink{AllowCompression}{AllowCompression} option.
\item [compression=LZ4] \hfill \\
\index[dir]{compression}
\index[dir]{Directive!compression}
All files saved will be software compressed using the LZ4
compression format. The compression is done on a file by file basis by
the File daemon. Everything else about GZIP is true for LZ4.
LZ4 provides much faster compression and decompression speed but lower
compression ratio than GZIP. If your CPU is fast enough you should be able
to compress your data without making the backup duration longer.
Both LZ4 and LZ4HC have the same decompression speed which is about twice
the speed of the LZO compression. So for a restore both LZ4 and LZ4HC are
good candidates.
You can overwrite this option per Storage resource with
\ilink{AllowCompression}{AllowCompression} option.
\item [compression=LZ4HC] \hfill \\
\index[dir]{compression}
\index[dir]{Directive!compression}
All files saved will be software compressed using the LZ4HC
compression format. The compression is done on a file by file basis by
the File daemon. Everything else about GZIP is true for LZ4.
LZ4HC is the High Compression version of the LZ4 compression. It has
a higher compression ratio than LZ4 and is more comparable to GZIP-6
in both compression rate and cpu usage.
Both LZ4 and LZ4HC have the same decompression speed which is about twice
the speed of the LZO compression. So for a restore both LZ4 and LZ4HC are
good candidates.
You can overwrite this option per Storage resource with
\ilink{AllowCompression}{AllowCompression} option.
\end{description}
\item [signature={\textless}SHA1{\textbar}MD5{\textgreater}] \hfill \\
\begin{description}
\item [signature=SHA1] \hfill \\
\index[dir]{signature}
\index[dir]{SHA1}
\index[dir]{Directive!signature}
An SHA1 signature will be computed for all The SHA1 algorithm is
purported to be some what slower than the MD5 algorithm, but at the same
time is significantly better from a cryptographic point of view (i.e.
much fewer collisions, much lower probability of being hacked.) It adds
four more bytes than the MD5 signature. We strongly recommend that
either this option or MD5 be specified as a default for all files.
Note, only one of the two options MD5 or SHA1 can be computed for any
file.
\item [signature=MD5] \hfill \\
\index[dir]{signature}
\index[dir]{MD5}
\index[dir]{Directive!signature}
An MD5 signature will be computed for all files saved. Adding this
option generates about 5\% extra overhead for each file saved. In
addition to the additional CPU time, the MD5 signature adds 16 more
bytes per file to your catalog. We strongly recommend that this option
or the SHA1 option be specified as a default for all files.
\end{description}
\item[basejob={\textless}options{\textgreater}]
\index[dir]{basejob}
\index[dir]{Directive!basejob}
The options letters specified are used when running a {\bf Backup Level=Full}
with BaseJobs. The options letters are the same than in the \textbf{verify=}
option below.
\item[accurate={\textless}options{\textgreater}] \index[dir]{accurate}
\index[dir]{Directive!accurate} The options letters specified are used when
running a {\bf Backup Level=Incremental/Differential} in Accurate mode. The
options letters are the same than in the \textbf{verify=} option below.
\item [verify={\textless}options{\textgreater}] \hfill \\
\index[dir]{verify}
\index[dir]{Directive!verify}
The options letters specified are used when running a {\bf Verify
Level=Catalog} as well as the {\bf DiskToCatalog} level job. The options
letters may be any combination of the following:
\begin{description}
\item {\bf i}
compare the inodes
\item {\bf p}
compare the permission bits
\item {\bf n}
compare the number of links
\item {\bf u}
compare the user id
\item {\bf g}
compare the group id
\item {\bf s}
compare the size
\item {\bf a}
compare the access time
\item {\bf m}
compare the modification time (st\_mtime)
\item {\bf c}
compare the change time (st\_ctime)
\item {\bf d}
report file size decreases
\item {\bf 5}
compare the MD5 signature
\item {\bf 1}
compare the SHA1 signature
\item {\bf A}
Only for Accurate option, it allows to always backup the file
\end{description}
A useful set of general options on the {\bf Level=Catalog} or {\bf
Level=DiskToCatalog} verify is {\bf pins5} i.e. compare permission bits,
inodes, number of links, size, and MD5 changes.
\item [onefs=yes{\textbar}no] \hfill \\
\index[dir]{onefs}
\index[dir]{Directive!onefs}
If set to {\bf yes} (the default), {\bf Bareos} will remain on a single
file system. That is it will not backup file systems that are mounted
on a subdirectory. If you are using a *nix system, you may not even be
aware that there are several different filesystems as they are often
automatically mounted by the OS (e.g. /dev, /net, /sys, /proc, ...).
Bareos will inform you when it decides not to
traverse into another filesystem. This can be very useful if you forgot
to backup a particular partition. An example of the informational
message in the job report is:
\footnotesize
\begin{verbatim}
rufus-fd: /misc is a different filesystem. Will not descend from / into /misc
rufus-fd: /net is a different filesystem. Will not descend from / into /net
rufus-fd: /var/lib/nfs/rpc_pipefs is a different filesystem. Will not descend from /var/lib/nfs into /var/lib/nfs/rpc_pipefs
rufus-fd: /selinux is a different filesystem. Will not descend from / into /selinux
rufus-fd: /sys is a different filesystem. Will not descend from / into /sys
rufus-fd: /dev is a different filesystem. Will not descend from / into /dev
rufus-fd: /home is a different filesystem. Will not descend from / into /home
\end{verbatim}
\normalsize
If you wish to backup multiple filesystems, you can explicitly
list each filesystem you want saved. Otherwise, if you set the onefs option
to {\bf no}, Bareos will backup all mounted file systems (i.e. traverse mount
points) that are found within the {\bf FileSet}. Thus if you have NFS or
Samba file systems mounted on a directory listed in your FileSet, they will
also be backed up. Normally, it is preferable to set {\bf onefs=yes} and to
explicitly name each filesystem you want backed up. Explicitly naming the
filesystems you want backed up avoids the possibility of getting into a
infinite loop recursing filesystems. Another possibility is to
use {\bf onefs=no} and to set {\bf fstype=ext2, ...}.
See the example below for more details.
If you think that Bareos should be backing up a particular directory
and it is not, and you have {\bf onefs=no} set, before you complain,
please do:
\footnotesize
\begin{verbatim}
stat /
stat <filesystem>
\end{verbatim}
\normalsize
where you replace {\bf filesystem} with the one in question. If the
{\bf Device:} number is different for / and for your filesystem, then they
are on different filesystems. E.g.
\footnotesize
\begin{verbatim}
stat /
File: `/'
Size: 4096 Blocks: 16 IO Block: 4096 directory
Device: 302h/770d Inode: 2 Links: 26
Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root)
Access: 2005-11-10 12:28:01.000000000 +0100
Modify: 2005-09-27 17:52:32.000000000 +0200
Change: 2005-09-27 17:52:32.000000000 +0200
stat /net
File: `/home'
Size: 4096 Blocks: 16 IO Block: 4096 directory
Device: 308h/776d Inode: 2 Links: 7
Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root)
Access: 2005-11-10 12:28:02.000000000 +0100
Modify: 2005-11-06 12:36:48.000000000 +0100
Change: 2005-11-06 12:36:48.000000000 +0100
\end{verbatim}
\normalsize
Also be aware that even if you include {\bf /home} in your list
of files to backup, as you most likely should, you will get the
informational message that "/home is a different filesystem" when
Bareos is processing the {\bf /} directory. This message does not
indicate an error. This message means that while examining the
{\bf File =} referred to in the second part of the message, Bareos will
not descend into the directory mentioned in the first part of the message.
However, it is possible that the separate filesystem will be backed up
despite the message. For example, consider the following FileSet:
\footnotesize
\begin{verbatim}
File = /
File = /var
\end{verbatim}
\normalsize
where {\bf /var} is a separate filesystem. In this example, you will get a
message saying that Bareos will not decend from {\bf /} into {\bf /var}. But
it is important to realise that Bareos will descend into {\bf /var} from the
second File directive shown above. In effect, the warning is bogus,
but it is supplied to alert you to possible omissions from your FileSet. In
this example, {\bf /var} will be backed up. If you changed the FileSet such
that it did not specify {\bf /var}, then {\bf /var} will not be backed up.
\item [honor nodump flag={\textless}yes{\textbar}no{\textgreater}] \hfill \\
\index[dir]{honornodumpflag}
\index[dir]{Directive!honornodumpflag}
If your file system supports the {\bf nodump} flag (e. g. most
BSD-derived systems) Bareos will honor the setting of the flag
when this option is set to {\bf yes}. Files having this flag set
will not be included in the backup and will not show up in the
catalog. For directories with the {\bf nodump} flag set recursion
is turned off and the directory will be listed in the catalog.
If the {\bf honor nodump flag} option is not defined
or set to {\bf no} every file and directory will be eligible for
backup.
\item [portable=yes{\textbar}no] \hfill \\
\index[dir]{portable}
\index[dir]{Directive!portable}
\label{portable}
If set to {\bf yes} (default is {\bf no}), the Bareos File daemon will
backup Win32 files in a portable format, but not all Win32 file
attributes will be saved and restored. By default, this option is set
to {\bf no}, which means that on Win32 systems, the data will be backed
up using Windows API calls and on WinNT/2K/XP, all the security and
ownership attributes will be properly backed up (and restored). However
this format is not portable to other systems -- e.g. Unix, Win95/98/Me.
When backing up Unix systems, this option is ignored, and unless you
have a specific need to have portable backups, we recommend accept the
default ({\bf no}) so that the maximum information concerning your files
is saved.
\item [recurse=yes{\textbar}no] \hfill \\
\index[dir]{recurse}
\index[dir]{Directive!recurse}
If set to {\bf yes} (the default), Bareos will recurse (or descend) into
all subdirectories found unless the directory is explicitly excluded
using an {\bf exclude} definition. If you set {\bf recurse=no}, Bareos
will save the subdirectory entries, but not descend into the
subdirectories, and thus will not save the files or directories
contained in the subdirectories. Normally, you will want the default
({\bf yes}).
\item [sparse=yes{\textbar}no] \hfill \\
\index[dir]{sparse}
\index[dir]{Directive!sparse}
Enable special code that checks for sparse files such as created by
ndbm. The default is {\bf no}, so no checks are made for sparse files.
You may specify {\bf sparse=yes} even on files that are not sparse file.
No harm will be done, but there will be a small additional overhead to
check for buffers of all zero, and if there is a 32K block of all zeros
(see below), that block will become a hole in the file, which
may not be desirable if the original file was not a sparse file.
{\bf Restrictions:} Bareos reads files in 32K buffers. If the whole
buffer is zero, it will be treated as a sparse block and not written to
tape. However, if any part of the buffer is non-zero, the whole buffer
will be written to tape, possibly including some disk sectors (generally
4098 bytes) that are all zero. As a consequence, Bareos's detection of
sparse blocks is in 32K increments rather than the system block size.
If anyone considers this to be a real problem, please send in a request
for change with the reason.
If you are not familiar with sparse files, an example is say a file
where you wrote 512 bytes at address zero, then 512 bytes at address 1
million. The operating system will allocate only two blocks, and the
empty space or hole will have nothing allocated. However, when you read
the sparse file and read the addresses where nothing was written, the OS
will return all zeros as if the space were allocated, and if you backup
such a file, a lot of space will be used to write zeros to the volume.
Worse yet, when you restore the file, all the previously empty space
will now be allocated using much more disk space. By turning on the
{\bf sparse} option, Bareos will specifically look for empty space in
the file, and any empty space will not be written to the Volume, nor
will it be restored. The price to pay for this is that Bareos must
search each block it reads before writing it. On a slow system, this
may be important. If you suspect you have sparse files, you should
benchmark the difference or set sparse for only those files that are
really sparse.
You probably should not use this option on files or raw disk devices
that are not really sparse files (i.e. have holes in them).
\item [readfifo=yes{\textbar}no] \hfill \\
\index[dir]{readfifo}
\index[dir]{Directive!readfifo}
\label{readfifo}
If enabled, tells the Client to read the data on a backup and write the
data on a restore to any FIFO (pipe) that is explicitly mentioned in the
FileSet. In this case, you must have a program already running that
writes into the FIFO for a backup or reads from the FIFO on a restore.
This can be accomplished with the {\bf RunBeforeJob} directive. If this
is not the case, Bareos will hang indefinitely on reading/writing the
FIFO. When this is not enabled (default), the Client simply saves the
directory entry for the FIFO.
Normally, when Bareos runs a RunBeforeJob, it waits until that
script terminates, and if the script accesses the FIFO to write
into it, the Bareos job will block and everything will stall.
However, Vladimir Stavrinov as supplied tip that allows this feature
to work correctly. He simply adds the following to the beginning
of the RunBeforeJob script:
\begin{verbatim}
exec > /dev/null
\end{verbatim}
\begin{bconfig}{FileSet with Fifo}
Include {
Options {
signature=SHA1
readfifo=yes
}
File = /home/abc/fifo
}
\end{bconfig}
This feature can be used to do a "hot" database backup.
You can use the {\bf RunBeforeJob} to create the fifo
and to start a program that dynamically reads your database and writes
it to the fifo. Bareos will then write it to the Volume.
During the restore operation, the inverse is true, after Bareos creates
the fifo if there was any data stored with it (no need to explicitly
list it or add any options), that data will be written back to the fifo.
As a consequence, if any such FIFOs exist in the fileset to be restored,
you must ensure that there is a reader program or Bareos will block, and
after one minute, Bareos will time out the write to the fifo and move on
to the next file.
If you are planing to use a Fifo for backup, better take a look to the \nameref{bpipe} section.
\item [noatime=yes{\textbar}no] \hfill \\
\index[dir]{noatime}
\index[dir]{Directive!noatime}
If enabled, and if your Operating System supports the O\_NOATIME file
open flag, Bareos will open all files to be backed up with this option.
It makes it possible to read a file without updating the inode atime
(and also without the inode ctime update which happens if you try to set
the atime back to its previous value). It also prevents a race
condition when two programs are reading the same file, but only one does
not want to change the atime. It's most useful for backup programs and
file integrity checkers (and Bareos can fit on both categories).
This option is particularly useful for sites where users are sensitive
to their MailBox file access time. It replaces both the {\bf keepatime}
option without the inconveniences of that option (see below).
If your Operating System does not support this option, it will be
silently ignored by Bareos.
\item [mtimeonly=yes{\textbar}no] \hfill \\
\index[dir]{mtimeonly}
\index[dir]{Directive!mtimeonly}
If enabled, tells the Client that the selection of files during
Incremental and Differential backups should based only on the st\_mtime
value in the stat() packet. The default is {\bf no} which means that
the selection of files to be backed up will be based on both the
st\_mtime and the st\_ctime values. In general, it is not recommended
to use this option.
\item [keepatime=yes{\textbar}no] \hfill \\
\index[dir]{keepatime}
\index[dir]{Directive!keepatime}
The default is {\bf no}. When enabled, Bareos will reset the st\_atime
(access time) field of files that it backs up to their value prior to
the backup. This option is not generally recommended as there are very
few programs that use st\_atime, and the backup overhead is increased
because of the additional system call necessary to reset the times.
However, for some files, such as mailboxes, when Bareos backs up the
file, the user will notice that someone (Bareos) has accessed the
file. In this, case keepatime can be useful.
(I'm not sure this works on Win32).
Note, if you use this feature, when Bareos resets the access time, the
change time (st\_ctime) will automatically be modified by the system,
so on the next incremental job, the file will be backed up even if
it has not changed. As a consequence, you will probably also want
to use {\bf mtimeonly = yes} as well as keepatime (thanks to
Rudolf Cejka for this tip).
\item [checkfilechanges=yes{\textbar}no] \hfill \\
\index[dir]{checkfilechanges}
\index[dir]{Directive!checkfilechanges}
On versions 2.0.4 or greater,
if enabled, the Client will check size, age of each file after
their backup to see if they have changed during backup. If time
or size mismatch, an error will raise.
\begin{verbatim}
zog-fd: Client1.2007-03-31_09.46.21 Error: /tmp/test mtime changed during backup.
\end{verbatim}
In general, it is recommended to use this option.
\item [hardlinks=yes{\textbar}no] \hfill \\
\index[dir]{hardlinks}
\index[dir]{Directive!hardlinks}
When enabled (default), this directive will cause hard links to be
backed up. However, the File daemon keeps track of hard linked files and
will backup the data only once. The process of keeping track of the
hard links can be quite expensive if you have lots of them (tens of
thousands or more). This doesn't occur on normal Unix systems, but if
you use a program like BackupPC, it can create hundreds of thousands, or
even millions of hard links. Backups become very long and the File daemon
will consume a lot of CPU power checking hard links. In such a case,
set {\bf hardlinks=no} and hard links will not be backed up. Note, using
this option will most likely backup more data and on a restore the file
system will not be restored identically to the original.
\item [wild={\textless}string{\textgreater}] \hfill \\
\index[dir]{wild}
\index[dir]{Directive!wild}
Specifies a wild-card string to be applied to the filenames and
directory names. Note, if {\bf Exclude} is not enabled, the wild-card
will select which files are to be included. If {\bf Exclude=yes} is
specified, the wild-card will select which files are to be excluded.
Multiple wild-card directives may be specified, and they will be applied
in turn until the first one that matches. Note, if you exclude a
directory, no files or directories below it will be matched.
You may want to test your expressions prior to running your
backup by using the bwild program. Please see the
\ilink{Utilities}{bwild} chapter of this manual for
more. You can also test your full FileSet definition by using
the \ilink{estimate}{estimate} command in the Console
chapter of this manual.
It is recommended to enclose the string in double quotes.
\item [wilddir={\textless}string{\textgreater}] \hfill \\
\index[dir]{wilddir}
\index[dir]{Directive!wilddir}
Specifies a wild-card string to be applied to directory names only. No
filenames will be matched by this directive. Note, if {\bf Exclude} is
not enabled, the wild-card will select directories to be
included. If {\bf Exclude=yes} is specified, the wild-card will select
which directories are to be excluded. Multiple wild-card directives may be
specified, and they will be applied in turn until the first one that
matches. Note, if you exclude a directory, no files or directories
below it will be matched.
It is recommended to enclose the string in double quotes.
You may want to test your expressions prior to running your
backup by using the bwild program. Please see the
\ilink{Utilities}{bwild} chapter of this manual for
more. You can also test your full FileSet definition by using
the \ilink{estimate}{estimate} command in the Console
chapter of this manual.
An example of excluding with the WildDir option on Win32 machines is
presented below.
\item [wildfile={\textless}string{\textgreater}] \hfill \\
\index[dir]{wildfile}
\index[dir]{Directive!wildfile}
Specifies a wild-card string to be applied to non-directories. That
is no directory entries will be matched by this directive.
However, note that the match is done against the full path and filename,
so your wild-card string must take into account that filenames
are preceded by the full path.
If {\bf Exclude}
is not enabled, the wild-card will select which files are to be
included. If {\bf Exclude=yes} is specified, the wild-card will select
which files are to be excluded. Multiple wild-card directives may be
specified, and they will be applied in turn until the first one that
matches.
It is recommended to enclose the string in double quotes.
You may want to test your expressions prior to running your
backup by using the bwild program. Please see the
\ilink{Utilities}{bwild} chapter of this manual for
more. You can also test your full FileSet definition by using
the \ilink{estimate}{estimate} command in the Console
chapter of this manual.
An example of excluding with the WildFile option on Win32 machines is
presented below.
\item [regex={\textless}string{\textgreater}] \hfill \\
\index[dir]{regex}
\index[dir]{Directive!regex}
\label{FileRegex}
Specifies a POSIX extended regular expression to be applied to the
filenames and directory names, which include the full path. If {\bf