This repository has been archived by the owner on Mar 19, 2021. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 16
/
catmaintenance.tex
743 lines (607 loc) · 27.7 KB
/
catmaintenance.tex
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
%%
%%
\chapter{Catalog Maintenance}
\label{CatMaintenanceChapter}
\index[general]{Maintenance!Catalog}
\index[general]{Catalog Maintenance}
Without proper setup and maintenance, your Catalog may continue to grow
indefinitely as you run Jobs and backup Files, and/or it may become
very inefficient and slow. How fast the size of your
Catalog grows depends on the number of Jobs you run and how many files they
backup. By deleting records within the database, you can make space available
for the new records that will be added during the next Job. By constantly
deleting old expired records (dates older than the Retention period), your
database size will remain constant.
\section{Setting Retention Periods}
\index[general]{Setting Retention Periods}
\index[general]{Periods!Setting Retention}
\label{Retention}
Bareos uses three Retention periods: the {\bf File Retention} period,
the {\bf Job Retention} period, and the {\bf Volume Retention} period. Of
these three, the File Retention period is by far the most important in
determining how large your database will become.
The {\bf File Retention} and the {\bf Job Retention} are specified in each
Client resource as is shown below. The {\bf Volume Retention} period is
specified in the Pool resource, and the details are given in the next chapter
of this manual.
\begin{description}
\item [File Retention = {\textless}time-period-specification{\textgreater}]
\index[general]{File Retention}
The File Retention record defines the length of time that Bareos will keep
File records in the Catalog database. When this time period expires, and if
{\bf AutoPrune} is set to {\bf yes}, Bareos will prune (remove) File records
that are older than the specified File Retention period. The pruning will
occur at the end of a backup Job for the given Client. Note that the Client
database record contains a copy of the File and Job retention periods, but
Bareos uses the current values found in the Director's Client resource to do
the pruning.
Since File records in the database account for probably 80 percent of the
size of the database, you should carefully determine exactly what File
Retention period you need. Once the File records have been removed from
the database, you will no longer be able to restore individual files
in a Job. However, as long as the
Job record still exists, you will be able to restore all files in the
job.
Retention periods are specified in seconds, but as a convenience, there are
a number of modifiers that permit easy specification in terms of minutes,
hours, days, weeks, months, quarters, or years on the record. See the
\ilink{ Configuration chapter}{Time} of this manual for additional details
of modifier specification.
The default File retention period is 60 days.
\item [Job Retention = {\textless}time-period-specification{\textgreater}]
\index[general]{Job Retention}
The Job Retention record defines the length of time that {\bf Bareos}
will keep Job records in the Catalog database. When this time period
expires, and if {\bf AutoPrune} is set to {\bf yes} Bareos will prune
(remove) Job records that are older than the specified Job Retention
period. Note, if a Job record is selected for pruning, all associated File
and JobMedia records will also be pruned regardless of the File Retention
period set. As a consequence, you normally will set the File retention
period to be less than the Job retention period.
As mentioned above, once the File records are removed from the database,
you will no longer be able to restore individual files from the Job.
However, as long as the Job record remains in the database, you will be
able to restore all the files backuped for the Job.
As a consequence, it is generally a good idea to retain the Job
records much longer than the File records.
The retention period is specified in seconds, but as a convenience, there
are a number of modifiers that permit easy specification in terms of
minutes, hours, days, weeks, months, quarters, or years. See the \ilink{
Configuration chapter}{Time} of this manual for additional details of
modifier specification.
The default Job Retention period is 180 days.
\item [AutoPrune = {\textless}yes/no{\textgreater}]
\index[general]{AutoPrune}
If AutoPrune is set to {\bf yes} (default), Bareos will automatically apply
the File retention period and the Job retention period for the Client at the
end of the Job.
If you turn this off by setting it to {\bf no}, your Catalog will grow each
time you run a Job.
\end{description}
\section{MySQL}
\label{CompactingMySQL}
\subsection{Compacting Your MySQL Database}
\index[general]{Database!Compacting Your MySQL}
\index[general]{Compacting Your MySQL Database}
\TODO{Alternative approaches are investigated (by Sebastian).}
Over time, as noted above, your database will tend to grow.
Even though Bareos regularly prunes files, {\bf MySQL} does not automatically
reuse the space, and instead continues growing.
It is recommended that you use the {\bf OPTIMIZE TABLE} and {\bf ANALYZE TABLE}
statements regularly. This is to make sure that all indices are up to date
and to recycle space inside the database files.
Please note that the database files are never shrunk by {\bf MySQL}.
You can do this via the {\bf mysqlcheck} command:
\footnotesize
\begin{verbatim}
mysqlcheck -a -o -A
\end{verbatim}
\normalsize
If you really need to shrink the database files, you need to recreate the database.
This only works if you use per-table tablespaces by setting the {\bf innodb\_file\_per\_table} configuration option.
See
\elink{http://dev.mysql.com/doc/refman/5.5/en/innodb-multiple-tablespaces.html}
{here} for details.
\footnotesize
\begin{verbatim}
mysqldump -f --opt bareos > bareos.sql
mysql bareos < bareos.sql
rm -f bareos.sql
\end{verbatim}
\normalsize
Depending on the size of your database, this will take more or less time and a
fair amount of disk space.
\label{DatabaseRepair}
\label{RepairingMySQL}
\subsection{Repairing Your MySQL Database}
\index[general]{Database!Repairing Your MySQL}
\index[general]{Repairing Your MySQL Database}
If you find that you are getting errors writing to your MySQL database, or
Bareos hangs each time it tries to access the database, you should consider
running MySQL's database check and repair routines. The program you need to
run depends on the type of database indexing you are using. If you are using
the default, you will probably want to use {\bf myisamchk}. For more details
on how to do this, please consult the MySQL document at:
\elink{
http://www.mysql.com/doc/en/Repair.html}
{http://www.mysql.com/doc/en/Repair.html}.
If the errors you are getting are simply SQL warnings, then you might try
running \command{bareos-dbcheck} before (or possibly after) using the MySQL database repair
program. It can clean up many of the orphaned record problems, and certain
other inconsistencies in the Bareos database.
A typical cause of MySQL database problems is if your partition fills. In
such a case, you will need to create additional space on the partition.
\subsection{MySQL Table is Full}
\index[general]{Database!MySQL Table is Full}
\index[general]{MySQL Table is Full}
If you are running into the error {\bf The table 'File' is full ...},
it is probably because on version 4.x MySQL, the table is limited by
default to a maximum size of 4 GB and you have probably run into
the limit. The solution can be found at:
\elink{http://dev.mysql.com/doc/refman/5.0/en/full-table.html}
{http://dev.mysql.com/doc/refman/5.0/en/full-table.html}
You can display the maximum length of your table with:
\footnotesize
\begin{verbatim}
mysql bareos
SHOW TABLE STATUS FROM bareos like "File";
\end{verbatim}
\normalsize
If the column labeled "Max\_data\_length" is around 4Gb, this is likely
to be the source of your problem, and you can modify it with:
\footnotesize
\begin{verbatim}
mysql bareos
ALTER TABLE File MAX_ROWS=281474976710656;
\end{verbatim}
\normalsize
Alternatively you can modify your /etc/my.conf file before creating the
Bareos tables, and in the [mysqld] section set:
\footnotesize
\begin{verbatim}
set-variable = myisam_data_pointer_size=6
\end{verbatim}
\normalsize
The above myisam data pointer size must be made before you create your
Bareos tables or it will have no effect.
The row and pointer size changes should already be the default on MySQL
version 5.x, so making these changes should only be necessary on MySQL 4.x
depending on the size of your catalog database.
\subsection{MySQL Server Has Gone Away}
\index[general]{Database!MySQL Server Has Gone Away}
\index[general]{MySQL Server Has Gone Away}
If you are having problems with the MySQL server disconnecting or with
messages saying that your MySQL server has gone away, then please read
the MySQL documentation, which can be found at:
\elink{http://dev.mysql.com/doc/refman/5.0/en/gone-away.html}
{http://dev.mysql.com/doc/refman/5.0/en/gone-away.html}
\subsection{MySQL Temporary Tables}
When doing backups with large numbers of files, MySQL creates some
temporary tables. When these tables are small they can be held in
system memory, but as they approach some size, they
spool off to disk. The default location for these temp tables is
/tmp. Once that space fills up, Bareos daemons such as the Storage
daemon doing spooling can get strange errors. E.g.
\footnotesize
\begin{verbatim}
Fatal error: spool.c:402 Spool data read error.
Fatal error: backup.c:892 Network send error to SD. ERR=Connection reset by
peer
\end{verbatim}
\normalsize
What you need to do is setup MySQL to use a different (larger) temp
directory, which can be set in the /etc/my.cnf with these variables
set:
\footnotesize
\begin{verbatim}
tmpdir=/path/to/larger/tmpdir
bdb_tmpdir=/path/to/larger/tmpdir
\end{verbatim}
\normalsize
\section{PostgreSQL}
\label{RepairingPSQL}
\subsection{Repairing Your PostgreSQL Database}
\index[general]{Database!Repairing Your PostgreSQL}
\index[general]{Repairing Your PostgreSQL Database}
The same considerations apply that are indicated above for MySQL. That is,
consult the PostgreSQL documents for how to repair the database, and also
consider using \command{bareos-dbcheck} program if the conditions are reasonable for
using (see above).
\subsection{Compacting Your PostgreSQL Database}
\index[general]{Database!Compacting Your PostgreSQL}
\index[general]{Compacting Your PostgreSQL Database}
\label{CompactingPostgres}
Over time, as noted above, your database will tend to grow. I've noticed that
even though Bareos regularly prunes files, PostgreSQL has a {\bf VACUUM}
command that will compact your database for you. Alternatively you may want to
use the {\bf vacuumdb} command, which can be run from a cron job.
All database programs have some means of writing the database out in ASCII
format and then reloading it. Doing so will re-create the database from
scratch producing a compacted result, so below, we show you how you can do
this for PostgreSQL.
For a {\bf PostgreSQL} database, you could write the Bareos database as an
ASCII file (bareos.sql) then reload it by doing the following:
\footnotesize
\begin{verbatim}
pg_dump -c bareos > bareos.sql
cat bareos.sql | psql bareos
rm -f bareos.sql
\end{verbatim}
\normalsize
Depending on the size of your database, this will take more or less time and a
fair amount of disk space. For example, you can {\bf cd} to the location of
the Bareos database (typically /usr/local/pgsql/data or possible
/var/lib/pgsql/data) and check the size.
There are certain PostgreSQL users who do not recommend the above
procedure. They have the following to say:
PostgreSQL does not
need to be dumped/restored to keep the database efficient. A normal
process of vacuuming will prevent the database from every getting too
large. If you want to fine-tweak the database storage, commands such
as VACUUM FULL, REINDEX, and CLUSTER exist specifically to keep you
from having to do a dump/restore.
Finally, you might want to look at the PostgreSQL documentation on
this subject at
\elink{http://www.postgresql.org/docs/8.1/interactive/maintenance.html}
{http://www.postgresql.org/docs/8.1/interactive/maintenance.html}.
% TODO: hidden, because outdated information. should we improve or remove it?
\hide{
\label{DatabasePerformance}
\section{Database Performance Issues}
\index[general]{Database Performance Issues}
\index[general]{Performance!Database}
There are a considerable number of ways each of the databases can be
tuned to improve the performance. Going from an untuned database to one
that is properly tuned can make a difference of a factor of 100 or more
in the time to insert or search for records.
For each of the databases, you may get significant improvements by adding
additional indexes. The comments in the Bareos make\_xxx\_tables give some
indications as to what indexes may be appropriate. Please see below
for specific instructions on checking indexes.
For MySQL, what is very important is to use the examine the
my.cnf file (usually in /etc/my.cnf).
You may obtain significant performances by switching to
the my-large.cnf or my-huge.cnf files that come with the MySQL source
code.
For SQLite3, one significant factor in improving the performance is
to ensure that there is a "PRAGMA synchronous = NORMAL;" statement.
This reduces the number of times that the database flushes the in memory
cache to disk. There are other settings for this PRAGMA that can
give even further performance improvements at the risk of a database
corruption if your system crashes.
For PostgreSQL, you might want to consider turning fsync off. Of course
doing so can cause corrupted databases in the event of a machine crash.
There are many different ways that you can tune PostgreSQL, the
following document discusses a few of them:
\elink{
http://www.varlena.com/varlena/GeneralBits/Tidbits/perf.html}
{http://www.varlena.com/varlena/GeneralBits/Tidbits/perf.html}.
There is also a PostgreSQL FAQ question number 3.3 that may
answer some of your questions about how to improve performance
of the PostgreSQL engine:
\elink{
http://www.postgresql.org/docs/faqs.FAQ.html\#3.3}
{http://www.postgresql.org/docs/faqs.FAQ.html\#3.3}.
% TODO: verify above is correct. is this okay for book?
Also for PostgreSQL, look at what "effective\_cache\_size". For a 2GB memory
machine, you probably want to set it at 131072, but don't set it too high.
In addition, for a 2GB system, work\_mem = 256000 and
maintenance\_work\_mem = 256000 seem to be reasonable values. Make
sure your checkpoint\_segments is set to at least 8.
}
\section{Performance Issues Indexes}
\index[general]{Database Performance Issues Indexes}
\index[general]{Performance!Database}
\TODO{Is chapter needs verification/updating (by Stephan).}
One of the most important considerations for improving performance on
the Bareos database is to ensure that it has all the appropriate indexes.
Several users have reported finding that their database did not have
all the indexes in the default configuration. In addition, you may
find that because of your own usage patterns, you need additional indexes.
The most important indexes for performance are the two indexes on the
{\bf File} table. The first index is on {\bf FileId} and is automatically
made because it is the unique key used to access the table. The other
one is the (JobId, PathId, Filename) index. If these Indexes
are not present, your performance may suffer a lot.
\subsection{PostgreSQL Indexes}
On PostgreSQL, you can check to see if you have the proper indexes using
the following commands:
\footnotesize
\begin{verbatim}
psql bareos
select * from pg_indexes where tablename='file';
\end{verbatim}
\normalsize
If you do not see output that indicates that all three indexes
are created, you can create the two additional indexes using:
\footnotesize
\begin{verbatim}
psql bareos
CREATE INDEX file_jobid_idx on file (jobid);
CREATE INDEX file_jpf_idx on file (jobid, pathid, filenameid);
\end{verbatim}
\normalsize
Make sure that you doesn't have an index on File (filenameid, pathid).
\subsection{MySQL Indexes}
On MySQL, you can check if you have the proper indexes by:
\footnotesize
\begin{verbatim}
mysql bareos
show index from File;
\end{verbatim}
\normalsize
If the indexes are not present, especially the JobId index, you can
create them with the following commands:
\footnotesize
\begin{verbatim}
mysql bareos
CREATE INDEX file_jobid_idx on File (JobId);
CREATE INDEX file_jpf_idx on File (JobId, FilenameId, PathId);
\end{verbatim}
\normalsize
Though normally not a problem, you should ensure that the indexes
defined for Filename and Path are both set to 255 characters. Some users
reported performance problems when their indexes were set to 50 characters.
To check, do:
\footnotesize
\begin{verbatim}
mysql bareos
show index from Filename;
show index from Path;
\end{verbatim}
\normalsize
and what is important is that for Filename, you have an index with
Key\_name "Name" and Sub\_part "255". For Path, you should have a Key\_name
"Path" and Sub\_part "255". If one or the other does not exist or the
Sub\_part is less that 255, you can drop and recreate the appropriate
index with:
\footnotesize
\begin{verbatim}
mysql bareos
DROP INDEX Path on Path;
CREATE INDEX Path on Path (Path(255));
DROP INDEX Name on Filename;
CREATE INDEX Name on Filename (Name(255));
\end{verbatim}
\normalsize
\subsection{SQLite Indexes}
On SQLite, you can check if you have the proper indexes by:
\footnotesize
\begin{verbatim}
sqlite <path>/bareos.db
select * from sqlite_master where type='index' and tbl_name='File';
\end{verbatim}
\normalsize
If the indexes are not present, especially the JobId index, you can
create them with the following commands:
\footnotesize
\begin{verbatim}
sqlite <path>/bareos.db
CREATE INDEX file_jobid_idx on File (JobId);
CREATE INDEX file_jfp_idx on File (JobId, PathId, FilenameId);
\end{verbatim}
\normalsize
% hidE: sqlite is not supported
\hide{
\section{Compacting Your SQLite Database}
\index[general]{Compacting Your SQLite Database}
\index[general]{Database!Compacting Your SQLite}
First please read the previous section that explains why it is necessary to
compress a database. SQLite version 2.8.4 and greater have the {\bf Vacuum}
command for compacting the database.
\footnotesize
\begin{verbatim}
cd {\bf working-directory}
echo 'vacuum;' | sqlite bareos.db
\end{verbatim}
\normalsize
As an alternative, you can use the following commands, adapted to your system:
\footnotesize
\begin{verbatim}
cd {\bf working-directory}
echo '.dump' | sqlite bareos.db > bareos.sql
rm -f bareos.db
sqlite bareos.db < bareos.sql
rm -f bareos.sql
\end{verbatim}
\normalsize
Where {\bf working-directory} is the directory that you specified in the
Director's configuration file. Note, in the case of SQLite, it is necessary to
completely delete (rm) the old database before creating a new compressed
version.
}
\section{Backing Up Your Bareos Database}
\index[general]{Backing Up Your Bareos Database}
\index[general]{Database!Backing Up Your Bareos}
\label{BackingUpBareos}
If ever the machine on which your Bareos database crashes, and you need to
restore from backup tapes, one of your first priorities will probably be to
recover the database. Although Bareos will happily backup your catalog
database if it is specified in the FileSet, this is not a very good way to do
it, because the database will be saved while Bareos is modifying it. Thus the
database may be in an instable state. Worse yet, you will backup the database
before all the Bareos updates have been applied.
To resolve these problems, you need to backup the database after all the backup
jobs have been run. In addition, you will want to make a copy while Bareos is
not modifying it. To do so, you can use two scripts provided in the release
{\bf make\_catalog\_backup} and {\bf delete\_catalog\_backup}. These files
will be automatically generated along with all the other Bareos scripts. The
first script will make an ASCII copy of your Bareos database into {\bf
bareos.sql} in the working directory you specified in your configuration, and
the second will delete the {\bf bareos.sql} file.
The basic sequence of events to make this work correctly is as follows:
\begin{itemize}
\item Run all your nightly backups
\item After running your nightly backups, run a Catalog backup Job
\item The Catalog backup job must be scheduled after your last nightly backup
\item You use {\bf RunBeforeJob} to create the ASCII backup file and {\bf
RunAfterJob} to clean up
\end{itemize}
Assuming that you start all your nightly backup jobs at 1:05 am (and that they
run one after another), you can do the catalog backup with the following
additional Director configuration statements:
\footnotesize
\begin{verbatim}
# Backup the catalog database (after the nightly save)
Job {
Name = "BackupCatalog"
Type = Backup
Client=rufus-fd
FileSet="Catalog"
Schedule = "WeeklyCycleAfterBackup"
Storage = DLTDrive
Messages = Standard
Pool = Default
# This creates an ASCII copy of the catalog
# Arguments to make_catalog_backup.pl are:
# make_catalog_backup.pl <catalog-name>
RunBeforeJob = "/usr/lib/bareos/scripts/make_catalog_backup.pl MyCatalog"
# This deletes the copy of the catalog
RunAfterJob = "/usr/lib/bareos/scripts/delete_catalog_backup"
# This sends the bootstrap via mail for disaster recovery.
# Should be sent to another system, please change recipient accordingly
Write Bootstrap = "|/usr/sbin/bsmtp -h localhost -f \"\(Bareos\) \" -s \"Bootstrap for Job %j\" root@localhost"
}
# This schedule does the catalog. It starts after the WeeklyCycle
Schedule {
Name = "WeeklyCycleAfterBackup
Run = Level=Full sun-sat at 1:10
}
# This is the backup of the catalog
FileSet {
Name = "Catalog"
Include {
Options {
signature=MD5
}
File = "/var/lib/bareos/bareos.sql" # database dump
File = "/etc/bareos" # configuration
}
}
\end{verbatim}
\normalsize
It is preferable
to write/send the bootstrap file to another computer. It will allow
you to quickly recover the database backup should that be necessary. If
you do not have a bootstrap file, it is still possible to recover your
database backup, but it will be more work and take longer.
\label{BackingUpBareosSecurityConsiderations}
\section{Security considerations}
\index[general]{Backing Up Your Bareos Database - Security Considerations}
\index[general]{Database!Backing Up Your Bareos Database - Security Considerations}
We provide make\_catalog\_backup as an example of what can be used to backup
your Bareos database. We expect you to take security precautions relevant
to your situation. make\_catalog\_backup is designed to take a password on
the command line. This is fine on machines with only trusted users. It is
not acceptable on machines without trusted users. Most database systems
provide a alternative method, which does not place the password on the
command line.
The make\_catalog\_backup script contains some warnings about how to use it. Please
read those tips.
To help you get started, we know PostgreSQL has a password file,
\elink{
.pgpass}{http://www.postgresql.org/docs/8.2/static/libpq-pgpass.html}, and
we know MySQL has
\elink{ .my.cnf}{http://dev.mysql.com/doc/refman/4.1/en/password-security.html}.
Only you can decide what is appropriate for your situation. We have provided
you with a starting point. We hope it helps.
\label{BackingUPOtherDBs}
\section{Backing Up Third Party Databases}
\index[general]{Backing Up Third Party Databases}
\index[general]{Databases!Backing Up Third Party}
If you are running a database in production mode on your machine, Bareos will
happily backup the files, but if the database is in use while Bareos is
reading it, you may back it up in an unstable state.
The best solution is to shutdown your database before backing it up, or use
some tool specific to your database to make a valid live copy perhaps by
dumping the database in ASCII format. I am not a database expert, so I cannot
provide you advice on how to do this, but if you are unsure about how to
backup your database, you might try visiting the Backup Central site, which
has been renamed Storage Mountain (www.backupcentral.com). In particular,
their
\elink{ Free Backup and Recovery
Software}{http://www.backupcentral.com/toc-free-backup-software.html} page has
links to scripts that show you how to shutdown and backup most major
databases.
\label{Size}
\section{Database Size}
\index[general]{Size!Database}
\index[general]{Database Size}
As mentioned above, if you do not do automatic pruning, your Catalog will grow
each time you run a Job. Normally, you should decide how long you want File
records to be maintained in the Catalog and set the {\bf File Retention}
period to that time. Then you can either wait and see how big your Catalog
gets or make a calculation assuming approximately 154 bytes for each File
saved and knowing the number of Files that are saved during each backup and
the number of Clients you backup.
For example, suppose you do a backup of two systems, each with 100,000 files.
Suppose further that you do a Full backup weekly and an Incremental every day,
and that the Incremental backup typically saves 4,000 files. The size of your
database after a month can roughly be calculated as:
\footnotesize
\begin{verbatim}
Size = 154 * No. Systems * (100,000 * 4 + 10,000 * 26)
\end{verbatim}
\normalsize
where we have assumed four weeks in a month and 26 incremental backups per month.
This would give the following:
\footnotesize
\begin{verbatim}
Size = 154 * 2 * (100,000 * 4 + 10,000 * 26)
or
Size = 308 * (400,000 + 260,000)
or
Size = 203,280,000 bytes
\end{verbatim}
\normalsize
So for the above two systems, we should expect to have a database size of
approximately 200 Megabytes. Of course, this will vary according to how many
files are actually backed up.
Below are some statistics for a MySQL database containing Job records for five
Clients beginning September 2001 through May 2002 (8.5 months) and File
records for the last 80 days. (Older File records have been pruned). For these
systems, only the user files and system files that change are backed up. The
core part of the system is assumed to be easily reloaded from the Red Hat rpms.
In the list below, the files (corresponding to Bareos Tables) with the
extension .MYD contain the data records whereas files with the extension .MYI
contain indexes.
You will note that the File records (containing the file attributes) make up
the large bulk of the number of records as well as the space used (459 Mega
Bytes including the indexes). As a consequence, the most important Retention
period will be the {\bf File Retention} period. A quick calculation shows that
for each File that is saved, the database grows by approximately 150 bytes.
\footnotesize
\begin{verbatim}
Size in
Bytes Records File
============ ========= ===========
168 5 Client.MYD
3,072 Client.MYI
344,394,684 3,080,191 File.MYD
115,280,896 File.MYI
2,590,316 106,902 Filename.MYD
3,026,944 Filename.MYI
184 4 FileSet.MYD
2,048 FileSet.MYI
49,062 1,326 JobMedia.MYD
30,720 JobMedia.MYI
141,752 1,378 Job.MYD
13,312 Job.MYI
1,004 11 Media.MYD
3,072 Media.MYI
1,299,512 22,233 Path.MYD
581,632 Path.MYI
36 1 Pool.MYD
3,072 Pool.MYI
5 1 Version.MYD
1,024 Version.MYI
\end{verbatim}
\normalsize
This database has a total size of approximately 450 Megabytes.
If we were using SQLite, the determination of the total database size would be
much easier since it is a single file, but we would have less insight to the
size of the individual tables as we have in this case.
Note, SQLite databases may be as much as 50\% larger than MySQL databases due
to the fact that all data is stored as ASCII strings. That is even binary
integers are stored as ASCII strings, and this seems to increase the space
needed.