-
Notifications
You must be signed in to change notification settings - Fork 4
/
user-guide-rhel.html
2987 lines (2948 loc) · 241 KB
/
user-guide-rhel.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head>
<title>pgBackRest User Guide - RHEL
</title><meta http-equiv="Content-Type" content="text/html;charset=utf-8"></meta>
<meta property="og:site_name" content="pgBackRest - Reliable PostgreSQL Backup & Restore"></meta>
<meta property="og:title" content="pgBackRest User Guide - RHEL"></meta>
<meta property="og:type" content="website"></meta>
<link rel="icon" href="favicon.png" type="image/png"></link>
<meta property="og:image:type" content="image/png"></meta>
<meta property="og:image" content="http://www.pgbackrest.org/logo.png"></meta>
<meta name="description" content="The pgBackRest User Guide demonstrates how to quickly and easily setup pgBackRest for your PostgreSQL database. Step-by-step instructions lead the user through all the important features of the fastest, most reliable PostgreSQL backup and restore solution."></meta>
<meta property="og:description" content="The pgBackRest User Guide demonstrates how to quickly and easily setup pgBackRest for your PostgreSQL database. Step-by-step instructions lead the user through all the important features of the fastest, most reliable PostgreSQL backup and restore solution."></meta>
<link rel="stylesheet" href="default.css" type="text/css"></link>
<script async src="https://www.googletagmanager.com/gtag/js?id=G-VKCRNV73H1"></script>
<script>window.dataLayer=window.dataLayer||[];function gtag(){dataLayer.push(arguments);}gtag('js',new Date());gtag('config','G-VKCRNV73H1');</script>
</head><body><div class="page-header"><div class="page-header-title">
pgBackRest User Guide
</div><div class="page-header-subtitle">
RHEL
</div></div><div class="page-menu"><div class="menu-body"><div class="menu"><a class="menu-link" href="/">
Home
</a></div><div class="menu"><a class="menu-link" href="user-guide-index.html">
User Guides
</a></div><div class="menu"><a class="menu-link" href="release.html">
Releases
</a></div><div class="menu"><a class="menu-link" href="configuration.html">
Configuration
</a></div><div class="menu"><a class="menu-link" href="command.html">
Commands
</a></div><div class="menu"><a class="menu-link" href="faq.html">
FAQ
</a></div><div class="menu"><a class="menu-link" href="metric.html">
Metrics
</a></div></div></div><div class="page-toc"><div class="page-toc-header"><div class="page-toc-title">
Table of Contents
</div></div><div class="page-toc-body"><div class="section1-toc"><div class="section1-toc-number"></div><div class="section1-toc-title"><a href="#introduction">
Introduction
</a></div></div><div class="section1-toc"><div class="section1-toc-number"></div><div class="section1-toc-title"><a href="#concept">
Concepts
</a></div><div class="section2-toc"><div class="section2-toc-number"></div><div class="section2-toc-title"><a href="#concept/backup">
Backup
</a></div></div><div class="section2-toc"><div class="section2-toc-number"></div><div class="section2-toc-title"><a href="#concept/restore">
Restore
</a></div></div><div class="section2-toc"><div class="section2-toc-number"></div><div class="section2-toc-title"><a href="#concept/wal">
Write Ahead Log (WAL)
</a></div></div><div class="section2-toc"><div class="section2-toc-number"></div><div class="section2-toc-title"><a href="#concept/encryption">
Encryption
</a></div></div></div><div class="section1-toc"><div class="section1-toc-number"></div><div class="section1-toc-title"><a href="#upgrading">
Upgrading pgBackRest
</a></div><div class="section2-toc"><div class="section2-toc-number"></div><div class="section2-toc-title"><a href="#upgrading/v1-v2">
Upgrading pgBackRest from v1 to v2
</a></div></div><div class="section2-toc"><div class="section2-toc-number"></div><div class="section2-toc-title"><a href="#upgrading/v2.x">
Upgrading pgBackRest from v2.x to v2.y
</a></div></div></div><div class="section1-toc"><div class="section1-toc-number"></div><div class="section1-toc-title"><a href="#build">
Build
</a></div></div><div class="section1-toc"><div class="section1-toc-number"></div><div class="section1-toc-title"><a href="#installation">
Installation
</a></div></div><div class="section1-toc"><div class="section1-toc-number"></div><div class="section1-toc-title"><a href="#quickstart">
Quick Start
</a></div><div class="section2-toc"><div class="section2-toc-number"></div><div class="section2-toc-title"><a href="#quickstart/setup-demo-cluster">
Setup Demo Cluster
</a></div></div><div class="section2-toc"><div class="section2-toc-number"></div><div class="section2-toc-title"><a href="#quickstart/configure-stanza">
Configure Cluster Stanza
</a></div></div><div class="section2-toc"><div class="section2-toc-number"></div><div class="section2-toc-title"><a href="#quickstart/create-repository">
Create the Repository
</a></div></div><div class="section2-toc"><div class="section2-toc-number"></div><div class="section2-toc-title"><a href="#quickstart/configure-archiving">
Configure Archiving
</a></div></div><div class="section2-toc"><div class="section2-toc-number"></div><div class="section2-toc-title"><a href="#quickstart/retention">
Configure Retention
</a></div></div><div class="section2-toc"><div class="section2-toc-number"></div><div class="section2-toc-title"><a href="#quickstart/configure-encryption">
Configure Repository Encryption
</a></div></div><div class="section2-toc"><div class="section2-toc-number"></div><div class="section2-toc-title"><a href="#quickstart/create-stanza">
Create the Stanza
</a></div></div><div class="section2-toc"><div class="section2-toc-number"></div><div class="section2-toc-title"><a href="#quickstart/check-configuration">
Check the Configuration
</a></div></div><div class="section2-toc"><div class="section2-toc-number"></div><div class="section2-toc-title"><a href="#quickstart/perform-backup">
Perform a Backup
</a></div></div><div class="section2-toc"><div class="section2-toc-number"></div><div class="section2-toc-title"><a href="#quickstart/schedule-backup">
Schedule a Backup
</a></div></div><div class="section2-toc"><div class="section2-toc-number"></div><div class="section2-toc-title"><a href="#quickstart/backup-info">
Backup Information
</a></div></div><div class="section2-toc"><div class="section2-toc-number"></div><div class="section2-toc-title"><a href="#quickstart/perform-restore">
Restore a Backup
</a></div></div></div><div class="section1-toc"><div class="section1-toc-number"></div><div class="section1-toc-title"><a href="#monitor">
Monitoring
</a></div><div class="section2-toc"><div class="section2-toc-number"></div><div class="section2-toc-title"><a href="#monitor/postgresql">
In <span class="postgres">PostgreSQL</span>
</a></div></div></div><div class="section1-toc"><div class="section1-toc-number"></div><div class="section1-toc-title"><a href="#backup">
Backup
</a></div><div class="section2-toc"><div class="section2-toc-number"></div><div class="section2-toc-title"><a href="#backup/bundle">
File Bundling
</a></div></div><div class="section2-toc"><div class="section2-toc-number"></div><div class="section2-toc-title"><a href="#backup/block">
Block Incremental
</a></div></div><div class="section2-toc"><div class="section2-toc-number"></div><div class="section2-toc-title"><a href="#backup/annotate">
Backup Annotations
</a></div></div></div><div class="section1-toc"><div class="section1-toc-number"></div><div class="section1-toc-title"><a href="#retention">
Retention
</a></div><div class="section2-toc"><div class="section2-toc-number"></div><div class="section2-toc-title"><a href="#retention/full">
Full Backup Retention
</a></div></div><div class="section2-toc"><div class="section2-toc-number"></div><div class="section2-toc-title"><a href="#retention/diff">
Differential Backup Retention
</a></div></div><div class="section2-toc"><div class="section2-toc-number"></div><div class="section2-toc-title"><a href="#retention/archive">
Archive Retention
</a></div></div></div><div class="section1-toc"><div class="section1-toc-number"></div><div class="section1-toc-title"><a href="#restore">
Restore
</a></div><div class="section2-toc"><div class="section2-toc-number"></div><div class="section2-toc-title"><a href="#restore/ownership">
File Ownership
</a></div></div><div class="section2-toc"><div class="section2-toc-number"></div><div class="section2-toc-title"><a href="#restore/option-delta">
Delta Option
</a></div></div><div class="section2-toc"><div class="section2-toc-number"></div><div class="section2-toc-title"><a href="#restore/option-db-include">
Restore Selected Databases
</a></div></div></div><div class="section1-toc"><div class="section1-toc-number"></div><div class="section1-toc-title"><a href="#pitr">
Point-in-Time Recovery
</a></div></div><div class="section1-toc"><div class="section1-toc-number"></div><div class="section1-toc-title"><a href="#delete-stanza">
Delete a Stanza
</a></div></div><div class="section1-toc"><div class="section1-toc-number"></div><div class="section1-toc-title"><a href="#multi-repo">
Multiple Repositories
</a></div></div><div class="section1-toc"><div class="section1-toc-number"></div><div class="section1-toc-title"><a href="#azure-support">
Azure-Compatible Object Store Support
</a></div></div><div class="section1-toc"><div class="section1-toc-number"></div><div class="section1-toc-title"><a href="#s3-support">
S3-Compatible Object Store Support
</a></div></div><div class="section1-toc"><div class="section1-toc-number"></div><div class="section1-toc-title"><a href="#sftp-support">
SFTP Support
</a></div></div><div class="section1-toc"><div class="section1-toc-number"></div><div class="section1-toc-title"><a href="#gcs-support">
GCS-Compatible Object Store Support
</a></div></div><div class="section1-toc"><div class="section1-toc-number"></div><div class="section1-toc-title"><a href="#repo-host">
Dedicated Repository Host
</a></div><div class="section2-toc"><div class="section2-toc-number"></div><div class="section2-toc-title"><a href="#repo-host/install">
Installation
</a></div></div><div class="section2-toc"><div class="section2-toc-number"></div><div class="section2-toc-title"><a href="#repo-host/config">
Configuration
</a></div></div><div class="section2-toc"><div class="section2-toc-number"></div><div class="section2-toc-title"><a href="#repo-host/setup-tls">
Setup TLS Server
</a></div></div><div class="section2-toc"><div class="section2-toc-number"></div><div class="section2-toc-title"><a href="#repo-host/stanza-create">
Create and Check Stanza
</a></div></div><div class="section2-toc"><div class="section2-toc-number"></div><div class="section2-toc-title"><a href="#repo-host/perform-backup">
Perform a Backup
</a></div></div><div class="section2-toc"><div class="section2-toc-number"></div><div class="section2-toc-title"><a href="#repo-host/perform-restore">
Restore a Backup
</a></div></div></div><div class="section1-toc"><div class="section1-toc-number"></div><div class="section1-toc-title"><a href="#parallel-backup-restore">
Parallel Backup / Restore
</a></div></div><div class="section1-toc"><div class="section1-toc-number"></div><div class="section1-toc-title"><a href="#start-stop">
Starting and Stopping
</a></div></div><div class="section1-toc"><div class="section1-toc-number"></div><div class="section1-toc-title"><a href="#replication">
Replication
</a></div><div class="section2-toc"><div class="section2-toc-number"></div><div class="section2-toc-title"><a href="#replication/installation">
Installation
</a></div></div><div class="section2-toc"><div class="section2-toc-number"></div><div class="section2-toc-title"><a href="#replication/hot-standby">
Hot Standby
</a></div></div><div class="section2-toc"><div class="section2-toc-number"></div><div class="section2-toc-title"><a href="#replication/streaming">
Streaming Replication
</a></div></div></div><div class="section1-toc"><div class="section1-toc-number"></div><div class="section1-toc-title"><a href="#multi-stanza">
Multiple Stanzas
</a></div><div class="section2-toc"><div class="section2-toc-number"></div><div class="section2-toc-title"><a href="#multi-stanza/installation">
Installation
</a></div></div><div class="section2-toc"><div class="section2-toc-number"></div><div class="section2-toc-title"><a href="#multi-stanza/configuration">
Configuration
</a></div></div><div class="section2-toc"><div class="section2-toc-number"></div><div class="section2-toc-title"><a href="#multi-stanza/setup-demo-cluster">
Setup Demo Cluster
</a></div></div><div class="section2-toc"><div class="section2-toc-number"></div><div class="section2-toc-title"><a href="#multi-stanza/create-stanza">
Create the Stanza and Check Configuration
</a></div></div></div><div class="section1-toc"><div class="section1-toc-number"></div><div class="section1-toc-title"><a href="#async-archiving">
Asynchronous Archiving
</a></div><div class="section2-toc"><div class="section2-toc-number"></div><div class="section2-toc-title"><a href="#async-archiving/async-archive-push">
Archive Push
</a></div></div><div class="section2-toc"><div class="section2-toc-number"></div><div class="section2-toc-title"><a href="#async-archiving/async-archive-get">
Archive Get
</a></div></div></div><div class="section1-toc"><div class="section1-toc-number"></div><div class="section1-toc-title"><a href="#standby-backup">
Backup from a Standby
</a></div></div><div class="section1-toc"><div class="section1-toc-number"></div><div class="section1-toc-title"><a href="#upgrade-stanza">
Upgrading <span class="postgres">PostgreSQL</span>
</a></div></div></div></div><div class="page-body"><div class="section1"><a id="introduction"></a><div class="section1-header"><div class="section1-number"></div><div class="section1-title">
Introduction
</div></div><div class="section-body"><div class="section-body-text">
This user guide is intended to be followed sequentially from beginning to end — each section depends on the last. For example, the <a href="#restore">Restore</a> section relies on setup that is performed in the <a href="#quickstart">Quick Start</a> section. Once <span class="backrest">pgBackRest</span> is up and running then skipping around is possible but following the user guide in order is recommended the first time through.
</div><div class="section-body-text">
Although the examples in this guide are targeted at <span class="host">RHEL 7-8</span> and <span class="postgres">PostgreSQL</span> 12, it should be fairly easy to apply the examples to any Unix distribution and <span class="postgres">PostgreSQL</span> version. The only OS-specific commands are those to create, start, stop, and drop <span class="postgres">PostgreSQL</span> clusters. The <span class="backrest">pgBackRest</span> commands will be the same on any Unix system though the location of the executable may vary. While <span class="backrest">pgBackRest</span> strives to operate consistently across versions of <span class="postgres">PostgreSQL</span>, there are subtle differences between versions of <span class="postgres">PostgreSQL</span> that may show up in this guide when illustrating certain examples, e.g. <span class="postgres">PostgreSQL</span> path/file names and settings.
</div><div class="section-body-text">
Configuration information and documentation for PostgreSQL can be found in the <span class="postgres">PostgreSQL</span> <a href="http://www.postgresql.org/docs/12/static/index.html">Manual</a>.
</div><div class="section-body-text">
A somewhat novel approach is taken to documentation in this user guide. Each command is run on a virtual machine when the documentation is built from the XML source. This means you can have a high confidence that the commands work correctly in the order presented. Output is captured and displayed below the command when appropriate. If the output is not included it is because it was deemed not relevant or was considered a distraction from the narrative.
</div><div class="section-body-text">
All commands are intended to be run as an unprivileged user that has sudo privileges for both the <span class="user">root</span> and <span class="user">postgres</span> users. It's also possible to run the commands directly as their respective users without modification and in that case the <span class="cmd">sudo</span> commands can be stripped off.
</div></div></div><div class="section1"><a id="concept"></a><div class="section1-header"><div class="section1-number"></div><div class="section1-title">
Concepts
</div></div><div class="section-body"><div class="section-body-text">
The following concepts are defined as they are relevant to <span class="backrest">pgBackRest</span>, <span class="postgres">PostgreSQL</span>, and this user guide.
</div><div class="section2"><a id="concept/backup"></a><div class="section2-header"><div class="section2-number"></div><div class="section2-title">
Backup
</div></div><div class="section-body"><div class="section-body-text">
A backup is a consistent copy of a database cluster that can be restored to recover from a hardware failure, to perform Point-In-Time Recovery, or to bring up a new standby.
</div><div class="section-body-text">
<b>Full Backup</b>: <span class="backrest">pgBackRest</span> copies the entire contents of the database cluster to the backup. The first backup of the database cluster is always a Full Backup. <span class="backrest">pgBackRest</span> is always able to restore a full backup directly. The full backup does not depend on any files outside of the full backup for consistency.
</div><div class="section-body-text">
<b>Differential Backup</b>: <span class="backrest">pgBackRest</span> copies only those database cluster files that have changed since the last full backup. <span class="backrest">pgBackRest</span> restores a differential backup by copying all of the files in the chosen differential backup and the appropriate unchanged files from the previous full backup. The advantage of a differential backup is that it requires less disk space than a full backup, however, the differential backup and the full backup must both be valid to restore the differential backup.
</div><div class="section-body-text">
<b>Incremental Backup</b>: <span class="backrest">pgBackRest</span> copies only those database cluster files that have changed since the last backup (which can be another incremental backup, a differential backup, or a full backup). As an incremental backup only includes those files changed since the prior backup, they are generally much smaller than full or differential backups. As with the differential backup, the incremental backup depends on other backups to be valid to restore the incremental backup. Since the incremental backup includes only those files since the last backup, all prior incremental backups back to the prior differential, the prior differential backup, and the prior full backup must all be valid to perform a restore of the incremental backup. If no differential backup exists then all prior incremental backups back to the prior full backup, which must exist, and the full backup itself must be valid to restore the incremental backup.
</div></div></div><div class="section2"><a id="concept/restore"></a><div class="section2-header"><div class="section2-number"></div><div class="section2-title">
Restore
</div></div><div class="section-body"><div class="section-body-text">
A restore is the act of copying a backup to a system where it will be started as a live database cluster. A restore requires the backup files and one or more WAL segments in order to work correctly.
</div></div></div><div class="section2"><a id="concept/wal"></a><div class="section2-header"><div class="section2-number"></div><div class="section2-title">
Write Ahead Log (WAL)
</div></div><div class="section-body"><div class="section-body-text">
WAL is the mechanism that <span class="postgres">PostgreSQL</span> uses to ensure that no committed changes are lost. Transactions are written sequentially to the WAL and a transaction is considered to be committed when those writes are flushed to disk. Afterwards, a background process writes the changes into the main database cluster files (also known as the heap). In the event of a crash, the WAL is replayed to make the database consistent.
</div><div class="section-body-text">
WAL is conceptually infinite but in practice is broken up into individual 16MB files called segments. WAL segments follow the naming convention <span class="id">0000000100000A1E000000FE</span> where the first 8 hexadecimal digits represent the timeline and the next 16 digits are the logical sequence number (LSN).
</div></div></div><div class="section2"><a id="concept/encryption"></a><div class="section2-header"><div class="section2-number"></div><div class="section2-title">
Encryption
</div></div><div class="section-body"><div class="section-body-text">
Encryption is the process of converting data into a format that is unrecognizable unless the appropriate password (also referred to as passphrase) is provided.
</div><div class="section-body-text">
<span class="backrest">pgBackRest</span> will encrypt the repository based on a user-provided password, thereby preventing unauthorized access to data stored within the repository.
</div></div></div></div></div><div class="section1"><a id="upgrading"></a><div class="section1-header"><div class="section1-number"></div><div class="section1-title">
Upgrading pgBackRest
</div></div><div class="section-body"><div class="section2"><a id="upgrading/v1-v2"></a><div class="section2-header"><div class="section2-number"></div><div class="section2-title">
Upgrading pgBackRest from v1 to v2
</div></div><div class="section-body"><div class="section-body-text">
Upgrading from <span class="host">v1</span> to <span class="host">v2</span> is fairly straight-forward. The repository format has not changed and all non-deprecated options from <span class="host">v1</span> are accepted, so for most installations it is simply a matter of installing the new version.
</div><div class="section-body-text">
However, there are a few caveats:
</div><ul class="list-unordered"><li class="list-unordered">
The deprecated <span class="br-option">thread-max</span> option is no longer valid. Use <span class="br-option">process-max</span> instead.
</li><li class="list-unordered">
The deprecated <span class="br-option">archive-max-mb</span> option is no longer valid. This has been replaced with the <span class="br-option">archive-push-queue-max</span> option which has different semantics.
</li><li class="list-unordered">
The default for the <span class="br-option">backup-user</span> option has changed from <span class="id">backrest</span> to <span class="id">pgbackrest</span>.
</li><li class="list-unordered">
In <span class="host">v2.02</span> the default location of the <span class="backrest">pgBackRest</span> configuration file has changed from <span class="file">/etc/pgbackrest.conf</span> to <span class="file">/etc/pgbackrest/pgbackrest.conf</span>. If <span class="file">/etc/pgbackrest/pgbackrest.conf</span> does not exist, the <span class="file">/etc/pgbackrest.conf</span> file will be loaded instead, if it exists.
</li></ul><div class="section-body-text">
Many option names have changed to improve consistency although the old names from <span class="host">v1</span> are still accepted. In general, <span class="id">db-*</span> options have been renamed to <span class="id">pg-*</span> and <span class="id">backup-*</span>/<span class="id">retention-*</span> options have been renamed to <span class="id">repo-*</span> when appropriate.
</div><div class="section-body-text">
<span class="postgres">PostgreSQL</span> and repository options must be indexed when using the new names introduced in <span class="host">v2</span>, e.g. <span class="br-option">pg1-host</span>, <span class="br-option">pg1-path</span>, <span class="br-option">repo1-path</span>, <span class="br-option">repo1-type</span>, etc.
</div></div></div><div class="section2"><a id="upgrading/v2.x"></a><div class="section2-header"><div class="section2-number"></div><div class="section2-title">
Upgrading pgBackRest from v2.x to v2.y
</div></div><div class="section-body"><div class="section-body-text">
Upgrading from <span class="host">v2.x</span> to <span class="host">v2.y</span> is straight-forward. The repository format has not changed, so for most installations it is simply a matter of installing binaries for the new version. It is also possible to downgrade if you have not used new features that are unsupported by the older version.
</div></div></div></div></div><div class="section1"><a id="build"></a><div class="section1-header"><div class="section1-number"></div><div class="section1-title">
Build
</div></div><div class="section-body"><div class="section-body-text">
Installing <span class="backrest">pgBackRest</span> from a package is preferable to building from source. See <a href="#installation">Installation</a> for more information about packages.
</div><div class="section-body-text">
When building from source it is best to use a build host rather than building on production. Many of the tools required for the build should generally not be installed in production. <span class="backrest">pgBackRest</span> consists of a single executable so it is easy to copy to a new host once it is built.
</div><div class="section-body-text">
The preferred build method is <span class="host">meson</span>/<span class="host">ninja</span>, see <a href="user-guide.html#build">Build</a>. The <span class="host">autoconf</span>/<span class="host">make</span> method is shown here for legacy purposes.
</div><div class="execute"><div class="execute-title">
<span class="host">build</span> <b>⇒</b> Download version <span class="id">2.53</span> of <span class="backrest">pgBackRest</span> to <span class="path">/build</span> path
</div><div class="execute-body">
<pre class="execute-body-cmd">mkdir -p /build</pre>
<pre class="execute-body-cmd">wget -q -O - \
https://github.com/pgbackrest/pgbackrest/archive/release/2.53.tar.gz | \
tar zx -C /build</pre>
</div></div><div class="execute"><div class="execute-title">
<span class="host">build</span> <b>⇒</b> Install build dependencies
</div><div class="execute-body">
<pre class="execute-body-cmd">sudo yum install make gcc postgresql12-devel openssl-devel \
libxml2-devel lz4-devel libzstd-devel bzip2-devel libyaml-devel libssh2-devel</pre>
</div></div><div class="execute"><div class="execute-title">
<span class="host">build</span> <b>⇒</b> Configure and compile <span class="backrest">pgBackRest</span>
</div><div class="execute-body">
<pre class="execute-body-cmd">cd /build/pgbackrest-release-2.53/src && ./configure && make</pre>
</div></div></div></div><div class="section1"><a id="installation"></a><div class="section1-header"><div class="section1-number"></div><div class="section1-title">
Installation
</div></div><div class="section-body"><div class="section-body-text">
A new host named <span class="host">pg-primary</span> is created to contain the demo cluster and run <span class="backrest">pgBackRest</span> examples.
</div><div class="section-body-text">
Installing <span class="backrest">pgBackRest</span> from a package is preferable to building from source. When installing from a package the rest of the instructions in this section are generally not required, but it is possible that a package will skip creating one of the directories or apply incorrect permissions. In that case it may be necessary to manually create directories or update permissions.
</div><div class="section-body-text">
RHEL 7-8 packages for <span class="backrest">pgBackRest</span> are available from <a href="http://www.crunchydata.com">Crunchy Data</a> or <a href="http://yum.postgresql.org">yum.postgresql.org</a>.
</div><div class="section-body-text">
If packages are not provided for your distribution/version you can <a href="#build">build from source</a> and then install manually as shown here.
</div><div class="execute"><div class="execute-title">
<span class="host">pg-primary</span> <b>⇒</b> Install dependencies
</div><div class="execute-body">
<pre class="execute-body-cmd">sudo yum install postgresql-libs libssh2</pre>
</div></div><div class="execute"><div class="execute-title">
<span class="host">pg-primary</span> <b>⇒</b> Copy <span class="backrest">pgBackRest</span> binary from build host
</div><div class="execute-body">
<pre class="execute-body-cmd">sudo scp build:/build/pgbackrest-release-2.53/src/pgbackrest /usr/bin</pre>
<pre class="execute-body-cmd">sudo chmod 755 /usr/bin/pgbackrest</pre>
</div></div><div class="section-body-text">
<span class="backrest">pgBackRest</span> requires log and configuration directories and a configuration file.
</div><div class="execute"><div class="execute-title">
<span class="host">pg-primary</span> <b>⇒</b> Create <span class="backrest">pgBackRest</span> configuration file and directories
</div><div class="execute-body">
<pre class="execute-body-cmd">sudo mkdir -p -m 770 /var/log/pgbackrest</pre>
<pre class="execute-body-cmd">sudo chown postgres:postgres /var/log/pgbackrest</pre>
<pre class="execute-body-cmd">sudo mkdir -p /etc/pgbackrest</pre>
<pre class="execute-body-cmd">sudo mkdir -p /etc/pgbackrest/conf.d</pre>
<pre class="execute-body-cmd">sudo touch /etc/pgbackrest/pgbackrest.conf</pre>
<pre class="execute-body-cmd">sudo chmod 640 /etc/pgbackrest/pgbackrest.conf</pre>
<pre class="execute-body-cmd">sudo chown postgres:postgres /etc/pgbackrest/pgbackrest.conf</pre>
</div></div><div class="section-body-text">
<span class="backrest">pgBackRest</span> should now be properly installed but it is best to check. If any dependencies were missed then you will get an error when running <span class="backrest">pgBackRest</span> from the command line.
</div><div class="execute"><div class="execute-title">
<span class="host">pg-primary</span> <b>⇒</b> Make sure the installation worked
</div><div class="execute-body">
<pre class="execute-body-cmd">sudo -u postgres pgbackrest</pre>
<pre class="execute-body-output">pgBackRest 2.53 - General help
Usage:
pgbackrest [options] [command]
Commands:
annotate Add or modify backup annotation.
archive-get Get a WAL segment from the archive.
archive-push Push a WAL segment to the archive.
backup Backup a database cluster.
check Check the configuration.
expire Expire backups that exceed retention.
help Get help.
info Retrieve information about backups.
repo-get Get a file from a repository.
repo-ls List files in a repository.
restore Restore a database cluster.
server pgBackRest server.
server-ping Ping pgBackRest server.
stanza-create Create the required stanza data.
stanza-delete Delete a stanza.
stanza-upgrade Upgrade a stanza.
start Allow pgBackRest processes to run.
stop Stop pgBackRest processes from running.
verify Verify contents of the repository.
version Get version.
Use 'pgbackrest help [command]' for more information.</pre>
</div></div></div></div><div class="section1"><a id="quickstart"></a><div class="section1-header"><div class="section1-number"></div><div class="section1-title">
Quick Start
</div></div><div class="section-body"><div class="section-body-text">
The Quick Start section will cover basic configuration of <span class="backrest">pgBackRest</span> and <span class="postgres">PostgreSQL</span> and introduce the <span class="cmd">backup</span>, <span class="cmd">restore</span>, and <span class="cmd">info</span> commands.
</div><div class="section2"><a id="quickstart/setup-demo-cluster"></a><div class="section2-header"><div class="section2-number"></div><div class="section2-title">
Setup Demo Cluster
</div></div><div class="section-body"><div class="section-body-text">
Creating the demo cluster is optional but is strongly recommended, especially for new users, since the example commands in the user guide reference the demo cluster; the examples assume the demo cluster is running on the default port (i.e. 5432). The cluster will not be started until a later section because there is still some configuration to do.
</div><div class="execute"><div class="execute-title">
<span class="host">pg-primary</span> <b>⇒</b> Create the demo cluster
</div><div class="execute-body">
<pre class="execute-body-cmd">sudo -u postgres /usr/pgsql-12/bin/initdb \
-D /var/lib/pgsql/12/data -k -A peer</pre>
</div></div><div class="section-body-text">
By default RHEL 7-8 includes the day of the week in the log filename. This makes the user guide a bit more complicated so the <span class="pg-option">log_filename</span> is set to a constant.
</div><div class="config"><div class="config-title">
<span class="host">pg-primary</span>:<span class="file">/var/lib/pgsql/12/data/postgresql.conf</span> <b>⇒</b> Set <span class="pg-option">log_filename</span>
</div><div class="config-body"><div class="config-body-output">
log_filename = 'postgresql.log'
</div></div></div></div></div><div class="section2"><a id="quickstart/configure-stanza"></a><div class="section2-header"><div class="section2-number"></div><div class="section2-title">
Configure Cluster Stanza
</div></div><div class="section-body"><div class="section-body-text">
A stanza is the configuration for a <span class="postgres">PostgreSQL</span> database cluster that defines where it is located, how it will be backed up, archiving options, etc. Most db servers will only have one <span class="postgres">PostgreSQL</span> database cluster and therefore one stanza, whereas backup servers will have a stanza for every database cluster that needs to be backed up.<br/>
<br/>
It is tempting to name the stanza after the primary cluster but a better name describes the databases contained in the cluster. Because the stanza name will be used for the primary and all replicas it is more appropriate to choose a name that describes the actual function of the cluster, such as app or dw, rather than the local cluster name, such as main or prod.
</div><div class="section-body-text">
The name 'demo' describes the purpose of this cluster accurately so that will also make a good stanza name.
</div><div class="section-body-text">
<span class="backrest">pgBackRest</span> needs to know where the base data directory for the <span class="postgres">PostgreSQL</span> cluster is located. The path can be requested from <span class="postgres">PostgreSQL</span> directly but in a recovery scenario the <span class="postgres">PostgreSQL</span> process will not be available. During backups the value supplied to <span class="backrest">pgBackRest</span> will be compared against the path that <span class="postgres">PostgreSQL</span> is running on and they must be equal or the backup will return an error. Make sure that <span class="br-option">pg-path</span> is exactly equal to <span class="pg-option">data_directory</span> in <span class="file">postgresql.conf</span>.
</div><div class="section-body-text">
By default RHEL 7-8 stores clusters in <span class="path">/var/lib/pgsql/[version]/data</span> so it is easy to determine the correct path for the data directory.
</div><div class="section-body-text">
When creating the <span class="file">/etc/pgbackrest/pgbackrest.conf</span> file, the database owner (usually <span class="id">postgres</span>) must be granted read privileges.
</div><div class="config"><div class="config-title">
<span class="host">pg-primary</span>:<span class="file">/etc/pgbackrest/pgbackrest.conf</span> <b>⇒</b> Configure the <span class="postgres">PostgreSQL</span> cluster data directory
</div><div class="config-body"><div class="config-body-output">
[demo]<br/>
pg1-path=/var/lib/pgsql/12/data
</div></div></div><div class="section-body-text">
<span class="backrest">pgBackRest</span> configuration files follow the Windows INI convention. Sections are denoted by text in brackets and key/value pairs are contained in each section. Lines beginning with <span class="id">#</span> are ignored and can be used as comments.
</div><div class="section-body-text">
There are multiple ways the <span class="backrest">pgBackRest</span> configuration files can be loaded:
</div><ul class="list-unordered"><li class="list-unordered">
<span class="br-option">config</span> and <span class="br-option">config-include-path</span> are default: the default config file will be loaded, if it exists, and <span class="file">*.conf</span> files in the default config include path will be appended, if they exist.
</li><li class="list-unordered">
<span class="br-option">config</span> option is specified: only the specified config file will be loaded and is expected to exist.
</li><li class="list-unordered">
<span class="br-option">config-include-path</span> is specified: <span class="file">*.conf</span> files in the config include path will be loaded and the path is required to exist. The default config file will be be loaded if it exists. If it is desirable to load only the files in the specified config include path, then the <span class="br-option">--no-config</span> option can also be passed.
</li><li class="list-unordered">
<span class="br-option">config</span> and <span class="br-option">config-include-path</span> are specified: using the user-specified values, the config file will be loaded and <span class="file">*.conf</span> files in the config include path will be appended. The files are expected to exist.
</li><li class="list-unordered">
<span class="br-option">config-path</span> is specified: this setting will override the base path for the default location of the config file and/or the base path of the default config-include-path setting unless the config and/or config-include-path option is explicitly set.
</li></ul><div class="section-body-text">
The files are concatenated as if they were one big file; order doesn't matter, but there is precedence based on sections. The precedence (highest to lowest) is:
</div><ul class="list-unordered"><li class="list-unordered">
[<i>stanza</i>:<i>command</i>]
</li><li class="list-unordered">
[<i>stanza</i>]
</li><li class="list-unordered">
[global:<i>command</i>]
</li><li class="list-unordered">
[global]
</li></ul><div class="admonition"><div class="note">
NOTE:
</div><div class="note-text">
<span class="br-option">--config</span>, <span class="br-option">--config-include-path</span> and <span class="br-option">--config-path</span> are command-line only options.
</div></div><div class="section-body-text">
<span class="backrest">pgBackRest</span> can also be configured using environment variables as described in the <a href="command.html">command reference</a>.
</div><div class="execute"><div class="execute-title">
<span class="host">pg-primary</span> <b>⇒</b> Configure <span class="br-option">log-path</span> using the environment
</div><div class="execute-body">
<pre class="execute-body-cmd">sudo -u postgres bash -c ' \
export PGBACKREST_LOG_PATH=/path/set/by/env && \
pgbackrest --log-level-console=error help backup log-path'</pre>
<pre class="execute-body-output">pgBackRest 2.53 - 'backup' command - 'log-path' option help
Path where log files are stored.
The log path provides a location for pgBackRest to store log files. Note that
if log-level-file=off then no log path is required.
</pre>
<pre class="execute-body-output-highlight">current: /path/set/by/env</pre>
<pre class="execute-body-output">default: /var/log/pgbackrest</pre>
</div></div></div></div><div class="section2"><a id="quickstart/create-repository"></a><div class="section2-header"><div class="section2-number"></div><div class="section2-title">
Create the Repository
</div></div><div class="section-body"><div class="section-body-text">
The repository is where <span class="backrest">pgBackRest</span> stores backups and archives WAL segments.<br/>
<br/>
It may be difficult to estimate in advance how much space you'll need. The best thing to do is take some backups then record the size of different types of backups (full/incr/diff) and measure the amount of WAL generated per day. This will give you a general idea of how much space you'll need, though of course requirements will likely change over time as your database evolves.
</div><div class="section-body-text">
For this demonstration the repository will be stored on the same host as the <span class="postgres">PostgreSQL</span> server. This is the simplest configuration and is useful in cases where traditional backup software is employed to backup the database host.
</div><div class="execute"><div class="execute-title">
<span class="host">pg-primary</span> <b>⇒</b> Create the <span class="backrest">pgBackRest</span> repository
</div><div class="execute-body">
<pre class="execute-body-cmd">sudo mkdir -p /var/lib/pgbackrest</pre>
<pre class="execute-body-cmd">sudo chmod 750 /var/lib/pgbackrest</pre>
<pre class="execute-body-cmd">sudo chown postgres:postgres /var/lib/pgbackrest</pre>
</div></div><div class="section-body-text">
The repository path must be configured so <span class="backrest">pgBackRest</span> knows where to find it.
</div><div class="config"><div class="config-title">
<span class="host">pg-primary</span>:<span class="file">/etc/pgbackrest/pgbackrest.conf</span> <b>⇒</b> Configure the <span class="backrest">pgBackRest</span> repository path
</div><div class="config-body"><div class="config-body-output">
[demo]<br/>
pg1-path=/var/lib/pgsql/12/data<br/>
<br/>
[global]<br/>
repo1-path=/var/lib/pgbackrest
</div></div></div><div class="section-body-text">
Multiple repositories may also be configured. See <a href="#multi-repo">Multiple Repositories</a> for details.
</div></div></div><div class="section2"><a id="quickstart/configure-archiving"></a><div class="section2-header"><div class="section2-number"></div><div class="section2-title">
Configure Archiving
</div></div><div class="section-body"><div class="section-body-text">
Backing up a running <span class="postgres">PostgreSQL</span> cluster requires WAL archiving to be enabled. Note that <i>at least</i> one WAL segment will be created during the backup process even if no explicit writes are made to the cluster.
</div><div class="config"><div class="config-title">
<span class="host">pg-primary</span>:<span class="file">/var/lib/pgsql/12/data/postgresql.conf</span> <b>⇒</b> Configure archive settings
</div><div class="config-body"><div class="config-body-output">
archive_command = 'pgbackrest --stanza=demo archive-push %p'<br/>
archive_mode = on<br/>
log_filename = 'postgresql.log'<br/>
max_wal_senders = 3<br/>
wal_level = replica
</div></div></div><div class="section-body-text">
<span class="id">%p</span> is how <span class="postgres">PostgreSQL</span> specifies the location of the WAL segment to be archived. Setting <span class="pg-option">wal_level</span> to at least <span class="pg-setting">replica</span> and increasing <span class="pg-option">max_wal_senders</span> is a good idea even if there are currently no replicas as this will allow them to be added later without restarting the primary cluster.
</div><div class="section-body-text">
The <span class="postgres">PostgreSQL</span> cluster must be restarted after making these changes and before performing a backup.
</div><div class="execute"><div class="execute-title">
<span class="host">pg-primary</span> <b>⇒</b> Restart the demo cluster
</div><div class="execute-body">
<pre class="execute-body-cmd">sudo systemctl restart postgresql-12.service</pre>
</div></div><div class="section-body-text">
When archiving a WAL segment is expected to take more than 60 seconds (the default) to reach the <span class="backrest">pgBackRest</span> repository, then the <span class="backrest">pgBackRest</span> <span class="br-option">archive-timeout</span> option should be increased. Note that this option is not the same as the <span class="postgres">PostgreSQL</span> <span class="pg-option">archive_timeout</span> option which is used to force a WAL segment switch; useful for databases where there are long periods of inactivity. For more information on the <span class="postgres">PostgreSQL</span> <span class="pg-option">archive_timeout</span> option, see <span class="postgres">PostgreSQL</span> <a href="https://www.postgresql.org/docs/current/static/runtime-config-wal.html">Write Ahead Log</a>.
</div><div class="section-body-text">
The <span class="cmd">archive-push</span> command can be configured with its own options. For example, a lower compression level may be set to speed archiving without affecting the compression used for backups.
</div><div class="config"><div class="config-title">
<span class="host">pg-primary</span>:<span class="file">/etc/pgbackrest/pgbackrest.conf</span> <b>⇒</b> Config <span class="cmd">archive-push</span> to use a lower compression level
</div><div class="config-body"><div class="config-body-output">
[demo]<br/>
pg1-path=/var/lib/pgsql/12/data<br/>
<br/>
[global]<br/>
repo1-path=/var/lib/pgbackrest<br/>
<br/>
[global:archive-push]<br/>
compress-level=3
</div></div></div><div class="section-body-text">
This configuration technique can be used for any command and can even target a specific stanza, e.g. <span class="id">demo:archive-push</span>.
</div></div></div><div class="section2"><a id="quickstart/retention"></a><div class="section2-header"><div class="section2-number"></div><div class="section2-title">
Configure Retention
</div></div><div class="section-body"><div class="section-body-text">
<span class="backrest">pgBackRest</span> expires backups based on retention options.
</div><div class="config"><div class="config-title">
<span class="host">pg-primary</span>:<span class="file">/etc/pgbackrest/pgbackrest.conf</span> <b>⇒</b> Configure retention to 2 full backups
</div><div class="config-body"><div class="config-body-output">
[demo]<br/>
pg1-path=/var/lib/pgsql/12/data<br/>
<br/>
[global]<br/>
repo1-path=/var/lib/pgbackrest<br/>
repo1-retention-full=2<br/>
<br/>
[global:archive-push]<br/>
compress-level=3
</div></div></div><div class="section-body-text">
More information about retention can be found in the <a href="#retention">Retention</a> section.
</div></div></div><div class="section2"><a id="quickstart/configure-encryption"></a><div class="section2-header"><div class="section2-number"></div><div class="section2-title">
Configure Repository Encryption
</div></div><div class="section-body"><div class="section-body-text">
The repository will be configured with a cipher type and key to demonstrate encryption. Encryption is always performed client-side even if the repository type (e.g. <span class="host">S3</span> or other object store) supports encryption.
</div><div class="section-body-text">
It is important to use a long, random passphrase for the cipher key. A good way to generate one is to run: <span class="id">openssl rand -base64 48</span>.
</div><div class="config"><div class="config-title">
<span class="host">pg-primary</span>:<span class="file">/etc/pgbackrest/pgbackrest.conf</span> <b>⇒</b> Configure <span class="backrest">pgBackRest</span> repository encryption
</div><div class="config-body"><div class="config-body-output">
[demo]<br/>
pg1-path=/var/lib/pgsql/12/data<br/>
<br/>
[global]<br/>
repo1-cipher-pass=zWaf6XtpjIVZC5444yXB+cgFDFl7MxGlgkZSaoPvTGirhPygu4jOKOXf9LO4vjfO<br/>
repo1-cipher-type=aes-256-cbc<br/>
repo1-path=/var/lib/pgbackrest<br/>
repo1-retention-full=2<br/>
<br/>
[global:archive-push]<br/>
compress-level=3
</div></div></div><div class="section-body-text">
Once the repository has been configured and the stanza created and checked, the repository encryption settings cannot be changed.
</div></div></div><div class="section2"><a id="quickstart/create-stanza"></a><div class="section2-header"><div class="section2-number"></div><div class="section2-title">
Create the Stanza
</div></div><div class="section-body"><div class="section-body-text">
The <span class="cmd">stanza-create</span> command must be run to initialize the stanza. It is recommended that the <span class="cmd">check</span> command be run after <span class="cmd">stanza-create</span> to ensure archiving and backups are properly configured.
</div><div class="execute"><div class="execute-title">
<span class="host">pg-primary</span> <b>⇒</b> Create the stanza and check the configuration
</div><div class="execute-body">
<pre class="execute-body-cmd">sudo -u postgres pgbackrest --stanza=demo --log-level-console=info stanza-create</pre>
<pre class="execute-body-output">P00 INFO: stanza-create command begin 2.53: --exec-id=1139-e0615fda --log-level-console=info --no-log-timestamp --pg1-path=/var/lib/pgsql/12/data --repo1-cipher-pass=<redacted> --repo1-cipher-type=aes-256-cbc --repo1-path=/var/lib/pgbackrest --stanza=demo
P00 INFO: stanza-create for stanza 'demo' on repo1</pre>
<pre class="execute-body-output-highlight">P00 INFO: stanza-create command end: completed successfully</pre>
</div></div></div></div><div class="section2"><a id="quickstart/check-configuration"></a><div class="section2-header"><div class="section2-number"></div><div class="section2-title">
Check the Configuration
</div></div><div class="section-body"><div class="section-body-text">
The <span class="cmd">check</span> command validates that <span class="backrest">pgBackRest</span> and the <span class="pg-setting">archive_command</span> setting are configured correctly for archiving and backups for the specified stanza. It will attempt to check all repositories and databases that are configured for the host on which the command is run. It detects misconfigurations, particularly in archiving, that result in incomplete backups because required WAL segments did not reach the archive. The command can be run on the <span class="postgres">PostgreSQL</span> or repository host. The command may also be run on the standby host, however, since <span class="id">pg_switch_xlog()</span>/<span class="id">pg_switch_wal()</span> cannot be performed on the standby, the command will only test the repository configuration.<br/>
<br/>
Note that <span class="id">pg_create_restore_point('pgBackRest Archive Check')</span> and <span class="id">pg_switch_xlog()</span>/<span class="id">pg_switch_wal()</span> are called to force <span class="postgres">PostgreSQL</span> to archive a WAL segment.
</div><div class="execute"><div class="execute-title">
<span class="host">pg-primary</span> <b>⇒</b> Check the configuration
</div><div class="execute-body">
<pre class="execute-body-cmd">sudo -u postgres pgbackrest --stanza=demo --log-level-console=info check</pre>
<pre class="execute-body-output">P00 INFO: check command begin 2.53: --exec-id=1166-bb40d924 --log-level-console=info --no-log-timestamp --pg1-path=/var/lib/pgsql/12/data --repo1-cipher-pass=<redacted> --repo1-cipher-type=aes-256-cbc --repo1-path=/var/lib/pgbackrest --stanza=demo
P00 INFO: check repo1 configuration (primary)
P00 INFO: check repo1 archive for WAL (primary)</pre>
<pre class="execute-body-output-highlight">P00 INFO: WAL segment 000000010000000000000001 successfully archived to '/var/lib/pgbackrest/archive/demo/12-1/0000000100000000/000000010000000000000001-93862d8ad64ba4d162dd3ac0e272641a4ba344e5.gz' on repo1</pre>
<pre class="execute-body-output">P00 INFO: check command end: completed successfully</pre>
</div></div></div></div><div class="section2"><a id="quickstart/perform-backup"></a><div class="section2-header"><div class="section2-number"></div><div class="section2-title">
Perform a Backup
</div></div><div class="section-body"><div class="section-body-text">
By default <span class="backrest">pgBackRest</span> will wait for the next regularly scheduled checkpoint before starting a backup. Depending on the <span class="pg-option">checkpoint_timeout</span> and <span class="pg-option">checkpoint_segments</span> settings in <span class="postgres">PostgreSQL</span> it may be quite some time before a checkpoint completes and the backup can begin. Generally, it is best to set <span class="br-setting">start-fast=y</span> so that the backup starts immediately. This forces a checkpoint, but since backups are usually run once a day an additional checkpoint should not have a noticeable impact on performance. However, on very busy clusters it may be best to pass <span class="br-setting">--start-fast</span> on the command-line as needed.
</div><div class="config"><div class="config-title">
<span class="host">pg-primary</span>:<span class="file">/etc/pgbackrest/pgbackrest.conf</span> <b>⇒</b> Configure backup fast start
</div><div class="config-body"><div class="config-body-output">
[demo]<br/>
pg1-path=/var/lib/pgsql/12/data<br/>
<br/>
[global]<br/>
repo1-cipher-pass=zWaf6XtpjIVZC5444yXB+cgFDFl7MxGlgkZSaoPvTGirhPygu4jOKOXf9LO4vjfO<br/>
repo1-cipher-type=aes-256-cbc<br/>
repo1-path=/var/lib/pgbackrest<br/>
repo1-retention-full=2<br/>
start-fast=y<br/>
<br/>
[global:archive-push]<br/>
compress-level=3
</div></div></div><div class="section-body-text">
To perform a backup of the <span class="postgres">PostgreSQL</span> cluster run <span class="backrest">pgBackRest</span> with the <span class="cmd">backup</span> command.
</div><div class="execute"><div class="execute-title">
<span class="host">pg-primary</span> <b>⇒</b> Backup the demo cluster
</div><div class="execute-body">
<pre class="execute-body-cmd">sudo -u postgres pgbackrest --stanza=demo \
--log-level-console=info backup</pre>
<pre class="execute-body-output">P00 INFO: backup command begin 2.53: --exec-id=1236-5db31a10 --log-level-console=info --no-log-timestamp --pg1-path=/var/lib/pgsql/12/data --repo1-cipher-pass=<redacted> --repo1-cipher-type=aes-256-cbc --repo1-path=/var/lib/pgbackrest --repo1-retention-full=2 --stanza=demo --start-fast</pre>
<pre class="execute-body-output-highlight">P00 WARN: no prior backup exists, incr backup has been changed to full</pre>
<pre class="execute-body-output">P00 INFO: execute non-exclusive backup start: backup begins after the requested immediate checkpoint completes
P00 INFO: backup start archive = 000000010000000000000002, lsn = 0/2000028
[filtered 3 lines of output]
P00 INFO: check archive for segment(s) 000000010000000000000002:000000010000000000000003
P00 INFO: new backup label = 20240722-012645F</pre>
<pre class="execute-body-output-highlight">P00 INFO: full backup size = 23.5MB, file total = 981</pre>
<pre class="execute-body-output">P00 INFO: backup command end: completed successfully
P00 INFO: expire command begin 2.53: --exec-id=1236-5db31a10 --log-level-console=info --no-log-timestamp --repo1-cipher-pass=<redacted> --repo1-cipher-type=aes-256-cbc --repo1-path=/var/lib/pgbackrest --repo1-retention-full=2 --stanza=demo</pre>
</div></div><div class="section-body-text">
By default <span class="backrest">pgBackRest</span> will attempt to perform an incremental backup. However, an incremental backup must be based on a full backup and since no full backup existed <span class="backrest">pgBackRest</span> ran a full backup instead.
</div><div class="section-body-text">
The <span class="br-option">type</span> option can be used to specify a full or differential backup.
</div><div class="execute"><div class="execute-title">
<span class="host">pg-primary</span> <b>⇒</b> Differential backup of the demo cluster
</div><div class="execute-body">
<pre class="execute-body-cmd">sudo -u postgres pgbackrest --stanza=demo --type=diff \
--log-level-console=info backup</pre>
<pre class="execute-body-output"> [filtered 7 lines of output]
P00 INFO: check archive for segment(s) 000000010000000000000004:000000010000000000000005
P00 INFO: new backup label = 20240722-012645F_20240722-012649D</pre>
<pre class="execute-body-output-highlight">P00 INFO: diff backup size = 8.8KB, file total = 981</pre>
<pre class="execute-body-output">P00 INFO: backup command end: completed successfully
P00 INFO: expire command begin 2.53: --exec-id=1295-e6584353 --log-level-console=info --no-log-timestamp --repo1-cipher-pass=<redacted> --repo1-cipher-type=aes-256-cbc --repo1-path=/var/lib/pgbackrest --repo1-retention-full=2 --stanza=demo</pre>
</div></div><div class="section-body-text">
This time there was no warning because a full backup already existed. While incremental backups can be based on a full <i>or</i> differential backup, differential backups must be based on a full backup. A full backup can be performed by running the <span class="cmd">backup</span> command with <span class="br-setting">--type=full</span>.
</div><div class="section-body-text">
During an online backup <span class="backrest">pgBackRest</span> waits for WAL segments that are required for backup consistency to be archived. This wait time is governed by the <span class="backrest">pgBackRest</span> <span class="br-option">archive-timeout</span> option which defaults to 60 seconds. If archiving an individual segment is known to take longer then this option should be increased.
</div></div></div><div class="section2"><a id="quickstart/schedule-backup"></a><div class="section2-header"><div class="section2-number"></div><div class="section2-title">
Schedule a Backup
</div></div><div class="section-body"><div class="section-body-text">
Backups can be scheduled with utilities such as cron.
</div><div class="section-body-text">
In the following example, two cron jobs are configured to run; full backups are scheduled for 6:30 AM every Sunday with differential backups scheduled for 6:30 AM Monday through Saturday. If this crontab is installed for the first time mid-week, then pgBackRest will run a full backup the first time the differential job is executed, followed the next day by a differential backup.
</div>
<pre class="code-block">#m h dom mon dow command
30 06 * * 0 pgbackrest --type=full --stanza=demo backup
30 06 * * 1-6 pgbackrest --type=diff --stanza=demo backup</pre>
<div class="section-body-text">
Once backups are scheduled it's important to configure retention so backups are expired on a regular schedule, see <a href="#retention">Retention</a>.
</div></div></div><div class="section2"><a id="quickstart/backup-info"></a><div class="section2-header"><div class="section2-number"></div><div class="section2-title">
Backup Information
</div></div><div class="section-body"><div class="section-body-text">
Use the <span class="cmd">info</span> command to get information about backups.
</div><div class="execute"><div class="execute-title">
<span class="host">pg-primary</span> <b>⇒</b> Get info for the demo cluster
</div><div class="execute-body">
<pre class="execute-body-cmd">sudo -u postgres pgbackrest info</pre>
<pre class="execute-body-output">stanza: demo
status: ok
cipher: aes-256-cbc
db (current)
wal archive min/max (12): 000000010000000000000001/000000010000000000000005
</pre>
<pre class="execute-body-output-highlight"> full backup: 20240722-012645F</pre>
<pre class="execute-body-output"> timestamp start/stop: 2024-07-22 01:26:45+00 / 2024-07-22 01:26:48+00
wal start/stop: 000000010000000000000002 / 000000010000000000000003
database size: 23.5MB, database backup size: 23.5MB
repo1: backup set size: 2.8MB, backup size: 2.8MB
</pre>
<pre class="execute-body-output-highlight"> diff backup: 20240722-012645F_20240722-012649D</pre>
<pre class="execute-body-output"> timestamp start/stop: 2024-07-22 01:26:49+00 / 2024-07-22 01:26:51+00
wal start/stop: 000000010000000000000004 / 000000010000000000000005
database size: 23.5MB, database backup size: 8.8KB
repo1: backup set size: 2.8MB, backup size: 704B
backup reference list: 20240722-012645F</pre>
</div></div><div class="section-body-text">
The <span class="cmd">info</span> command operates on a single stanza or all stanzas. Text output is the default and gives a human-readable summary of backups for the stanza(s) requested. This format is subject to change with any release.<br/>
<br/>
For machine-readable output use <span class="br-option">--output=json</span>. The JSON output contains far more information than the text output and is kept stable unless a bug is found.<br/>
<br/>
Each stanza has a separate section and it is possible to limit output to a single stanza with the <span class="br-option">--stanza</span> option. The stanza '<span class="id">status</span>' gives a brief indication of the stanza's health. If this is '<span class="id">ok</span>' then <span class="backrest">pgBackRest</span> is functioning normally. If there are multiple repositories, then a status of '<span class="id">mixed</span>' indicates that the stanza is not in a healthy state on one or more of the repositories; in this case the state of the stanza will be detailed per repository. For cases in which an error on a repository occurred that is not one of the known error codes, then an error code of '<span class="id">other</span>' will be used and the full error details will be provided. The '<span class="id">wal archive min/max</span>' shows the minimum and maximum WAL currently stored in the archive and, in the case of multiple repositories, will be reported across all repositories unless the <span class="br-option">--repo</span> option is set. Note that there may be gaps due to archive retention policies or other reasons.<br/>
<br/>
The '<span class="id">backup/expire running</span>' message will appear beside the '<span class="id">status</span>' information if one of those commands is currently running on the host.<br/>
<br/>
The backups are displayed oldest to newest. The oldest backup will <i>always</i> be a full backup (indicated by an <span class="id">F</span> at the end of the label) but the newest backup can be full, differential (ends with <span class="id">D</span>), or incremental (ends with <span class="id">I</span>).<br/>
<br/>
The '<span class="id">timestamp start/stop</span>' defines the time period when the backup ran. The '<span class="id">timestamp stop</span>' can be used to determine the backup to use when performing Point-In-Time Recovery. More information about Point-In-Time Recovery can be found in the <a href="#pitr">Point-In-Time Recovery</a> section.<br/>
<br/>
The '<span class="id">wal start/stop</span>' defines the WAL range that is required to make the database consistent when restoring. The <span class="cmd">backup</span> command will ensure that this WAL range is in the archive before completing.<br/>
<br/>
The '<span class="id">database size</span>' is the full uncompressed size of the database while '<span class="id">database backup size</span>' is the amount of data in the database to actually back up (these will be the same for full backups).<br/>
<br/>
The '<span class="id">repo</span>' indicates in which repository this backup resides. The '<span class="id">backup set size</span>' includes all the files from this backup and any referenced backups in the repository that are required to restore the database from this backup while '<span class="id">backup size</span>' includes only the files in this backup (these will also be the same for full backups). Repository sizes reflect compressed file sizes if compression is enabled in <span class="backrest">pgBackRest</span>.<br/>
<br/>
The '<span class="id">backup reference list</span>' contains the additional backups that are required to restore this backup.
</div></div></div><div class="section2"><a id="quickstart/perform-restore"></a><div class="section2-header"><div class="section2-number"></div><div class="section2-title">
Restore a Backup
</div></div><div class="section-body"><div class="section-body-text">
Backups can protect you from a number of disaster scenarios, the most common of which are hardware failure and data corruption. The easiest way to simulate data corruption is to remove an important <span class="postgres">PostgreSQL</span> cluster file.
</div><div class="execute"><div class="execute-title">
<span class="host">pg-primary</span> <b>⇒</b> Stop the demo cluster and delete the <span class="file">pg_control</span> file
</div><div class="execute-body">
<pre class="execute-body-cmd">sudo systemctl stop postgresql-12.service</pre>
<pre class="execute-body-cmd">sudo -u postgres rm /var/lib/pgsql/12/data/global/pg_control</pre>
</div></div><div class="section-body-text">
Starting the cluster without this important file will result in an error.
</div><div class="execute"><div class="execute-title">
<span class="host">pg-primary</span> <b>⇒</b> Attempt to start the corrupted demo cluster
</div><div class="execute-body">
<pre class="execute-body-cmd">sudo systemctl start postgresql-12.service</pre>
<pre class="execute-body-cmd">sudo systemctl status postgresql-12.service</pre>
<pre class="execute-body-output"> [filtered 12 lines of output]
Jul 22 01:26:52 pg-primary systemd[1]: postgresql-12.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
Jul 22 01:26:52 pg-primary systemd[1]: postgresql-12.service: Failed with result 'exit-code'.</pre>
<pre class="execute-body-output-highlight-error">Jul 22 01:26:52 pg-primary systemd[1]: Failed to start PostgreSQL 12 database server.</pre>
</div></div><div class="section-body-text">
To restore a backup of the <span class="postgres">PostgreSQL</span> cluster run <span class="backrest">pgBackRest</span> with the <span class="cmd">restore</span> command. The cluster needs to be stopped (in this case it is already stopped) and all files must be removed from the <span class="postgres">PostgreSQL</span> data directory.
</div><div class="execute"><div class="execute-title">
<span class="host">pg-primary</span> <b>⇒</b> Remove old files from demo cluster
</div><div class="execute-body">
<pre class="execute-body-cmd">sudo -u postgres find /var/lib/pgsql/12/data -mindepth 1 -delete</pre>
</div></div><div class="execute"><div class="execute-title">
<span class="host">pg-primary</span> <b>⇒</b> Restore the demo cluster and start <span class="postgres">PostgreSQL</span>
</div><div class="execute-body">
<pre class="execute-body-cmd">sudo -u postgres pgbackrest --stanza=demo restore</pre>
<pre class="execute-body-cmd">sudo systemctl start postgresql-12.service</pre>
</div></div><div class="section-body-text">
This time the cluster started successfully since the restore replaced the missing <span class="file">pg_control</span> file.
</div><div class="section-body-text">
More information about the <span class="cmd">restore</span> command can be found in the <a href="#restore">Restore</a> section.
</div></div></div></div></div><div class="section1"><a id="monitor"></a><div class="section1-header"><div class="section1-number"></div><div class="section1-title">
Monitoring
</div></div><div class="section-body"><div class="section-body-text">
Monitoring is an important part of any production system. There are many tools available and <span class="backrest">pgBackRest</span> can be monitored on any of them with a little work.
</div><div class="section-body-text">
<span class="backrest">pgBackRest</span> can output information about the repository in JSON format which includes a list of all backups for each stanza and WAL archive info.
</div><div class="section2"><a id="monitor/postgresql"></a><div class="section2-header"><div class="section2-number"></div><div class="section2-title">
In <span class="postgres">PostgreSQL</span>
</div></div><div class="section-body"><div class="section-body-text">
The <span class="postgres">PostgreSQL</span> <span class="id">COPY</span> command allows <span class="backrest">pgBackRest</span> info to be loaded into a table. The following example wraps that logic in a function that can be used to perform real-time queries.
</div><div class="execute"><div class="execute-title">
<span class="host">pg-primary</span> <b>⇒</b> Load <span class="backrest">pgBackRest</span> info function for <span class="postgres">PostgreSQL</span>
</div><div class="execute-body">
<pre class="execute-body-cmd">sudo -u postgres cat \
/var/lib/pgsql/pgbackrest/doc/example/pgsql-pgbackrest-info.sql</pre>
<pre class="execute-body-output">-- An example of monitoring pgBackRest from within PostgreSQL
--
-- Use copy to export data from the pgBackRest info command into the jsonb
-- type so it can be queried directly by PostgreSQL.
-- Create monitor schema
create schema monitor;
-- Get pgBackRest info in JSON format
create function monitor.pgbackrest_info()
returns jsonb AS $$
declare
data jsonb;
begin
-- Create a temp table to hold the JSON data
create temp table temp_pgbackrest_data (data text);
-- Copy data into the table directly from the pgBackRest info command
copy temp_pgbackrest_data (data)
from program
'pgbackrest --output=json info' (format text);
select replace(temp_pgbackrest_data.data, E'\n', '\n')::jsonb
into data
from temp_pgbackrest_data;
drop table temp_pgbackrest_data;
return data;
end $$ language plpgsql;</pre>
<pre class="execute-body-cmd">sudo -u postgres psql -f \
/var/lib/pgsql/pgbackrest/doc/example/pgsql-pgbackrest-info.sql</pre>
</div></div><div class="section-body-text">
Now the <span class="id">monitor.pgbackrest_info()</span> function can be used to determine the last successful backup time and archived WAL for a stanza.
</div><div class="execute"><div class="execute-title">
<span class="host">pg-primary</span> <b>⇒</b> Query last successful backup time and archived WAL
</div><div class="execute-body">
<pre class="execute-body-cmd">sudo -u postgres cat \
/var/lib/pgsql/pgbackrest/doc/example/pgsql-pgbackrest-query.sql</pre>
<pre class="execute-body-output">-- Get last successful backup for each stanza
--
-- Requires the monitor.pgbackrest_info function.
with stanza as
(
select data->'name' as name,
data->'backup'->(
jsonb_array_length(data->'backup') - 1) as last_backup,
data->'archive'->(
jsonb_array_length(data->'archive') - 1) as current_archive
from jsonb_array_elements(monitor.pgbackrest_info()) as data
)
select name,
to_timestamp(
(last_backup->'timestamp'->>'stop')::numeric) as last_successful_backup,
current_archive->>'max' as last_archived_wal
from stanza;</pre>
<pre class="execute-body-cmd">sudo -u postgres psql -f \
/var/lib/pgsql/pgbackrest/doc/example/pgsql-pgbackrest-query.sql</pre>
<pre class="execute-body-output"> name | last_successful_backup | last_archived_wal
--------+------------------------+--------------------------
"demo" | 2024-07-22 01:26:51+00 | 000000010000000000000005
(1 row)</pre>
</div></div></div></div></div></div><div class="section1"><a id="backup"></a><div class="section1-header"><div class="section1-number"></div><div class="section1-title">
Backup
</div></div><div class="section-body"><div class="section-body-text">
When multiple repositories are configured, <span class="backrest">pgBackRest</span> will backup to the highest priority repository (e.g. <span class="id">repo1</span>) unless the <span class="br-option">--repo</span> option is specified.<br/>
<br/>
<span class="backrest">pgBackRest</span> does not have a built-in scheduler so it's best to run it from cron or some other scheduling mechanism.<br/>
<br/>
See <a href="#quickstart/perform-backup">Perform a Backup</a> for more details and examples.
</div><div class="section2"><a id="backup/bundle"></a><div class="section2-header"><div class="section2-number"></div><div class="section2-title">
File Bundling
</div></div><div class="section-body"><div class="section-body-text">
Bundling files together in the repository saves time during the backup and some space in the repository. This is especially pronounced when the repository is stored on an object store such as <span class="host">S3</span>. Per-file creation time on object stores is higher and very small files might cost as much to store as larger files.
</div><div class="section-body-text">
The file bundling feature is enabled with the <span class="br-option">repo-bundle</span> option.
</div><div class="config"><div class="config-title">
<span class="host">pg-primary</span>:<span class="file">/etc/pgbackrest/pgbackrest.conf</span> <b>⇒</b> Configure <span class="br-option">repo1-bundle</span>
</div><div class="config-body"><div class="config-body-output">
[demo]<br/>
pg1-path=/var/lib/pgsql/12/data<br/>
<br/>
[global]<br/>
repo1-bundle=y<br/>
repo1-cipher-pass=zWaf6XtpjIVZC5444yXB+cgFDFl7MxGlgkZSaoPvTGirhPygu4jOKOXf9LO4vjfO<br/>
repo1-cipher-type=aes-256-cbc<br/>
repo1-path=/var/lib/pgbackrest<br/>
repo1-retention-full=2<br/>
start-fast=y<br/>
<br/>
[global:archive-push]<br/>
compress-level=3
</div></div></div><div class="section-body-text">
A full backup without file bundling will have 1000+ files in the backup path, but with bundling the total number of files is greatly reduced. An additional benefit is that zero-length files are not stored (except in the manifest), whereas in a normal backup each zero-length file is stored individually.
</div><div class="execute"><div class="execute-title">
<span class="host">pg-primary</span> <b>⇒</b> Perform a full backup
</div><div class="execute-body">
<pre class="execute-body-cmd">sudo -u postgres pgbackrest --stanza=demo --type=full backup</pre>
</div></div><div class="execute"><div class="execute-title">
<span class="host">pg-primary</span> <b>⇒</b> Check file total
</div><div class="execute-body">
<pre class="execute-body-cmd">sudo -u postgres find /var/lib/pgbackrest/backup/demo/latest/ -type f | wc -l</pre>
<pre class="execute-body-output">5</pre>
</div></div><div class="section-body-text">
The <span class="br-option">repo-bundle-size</span> and <span class="br-option">repo-bundle-limit</span> options can be used for tuning, though the defaults should be optimal in most cases.
</div><div class="section-body-text">
While file bundling is generally more efficient, the downside is that it is more difficult to manually retrieve files from the repository. It may not be ideal for deduplicated storage since each full backup will arrange files in the bundles differently. Lastly, file bundles cannot be resumed, so be careful not to set <span class="br-option">repo-bundle-size</span> too high.
</div></div></div><div class="section2"><a id="backup/block"></a><div class="section2-header"><div class="section2-number"></div><div class="section2-title">
Block Incremental
</div></div><div class="section-body"><div class="section-body-text">
Block incremental backups save space by only storing the parts of a file that have changed since the prior backup rather than storing the entire file.
</div><div class="section-body-text">
The block incremental feature is enabled with the <span class="br-option">repo-block</span> option and it works best when enabled for all backup types. File bundling must also be enabled.
</div><div class="config"><div class="config-title">
<span class="host">pg-primary</span>:<span class="file">/etc/pgbackrest/pgbackrest.conf</span> <b>⇒</b> Configure <span class="br-option">repo1-block</span>
</div><div class="config-body"><div class="config-body-output">
[demo]<br/>
pg1-path=/var/lib/pgsql/12/data<br/>
<br/>
[global]<br/>
repo1-block=y<br/>
repo1-bundle=y<br/>
repo1-cipher-pass=zWaf6XtpjIVZC5444yXB+cgFDFl7MxGlgkZSaoPvTGirhPygu4jOKOXf9LO4vjfO<br/>
repo1-cipher-type=aes-256-cbc<br/>
repo1-path=/var/lib/pgbackrest<br/>
repo1-retention-full=2<br/>
start-fast=y<br/>
<br/>
[global:archive-push]<br/>
compress-level=3
</div></div></div></div></div><div class="section2"><a id="backup/annotate"></a><div class="section2-header"><div class="section2-number"></div><div class="section2-title">
Backup Annotations
</div></div><div class="section-body"><div class="section-body-text">
Users can attach informative key/value pairs to the backup. This option may be used multiple times to attach multiple annotations.
</div><div class="execute"><div class="execute-title">
<span class="host">pg-primary</span> <b>⇒</b> Perform a full backup with annotations
</div><div class="execute-body">
<pre class="execute-body-cmd">sudo -u postgres pgbackrest --stanza=demo --annotation=source="demo backup" \
--annotation=key=value --type=full backup</pre>
</div></div><div class="section-body-text">
Annotations are output by the <span class="cmd">info</span> command text output when a backup is specified with <span class="br-option">--set</span> and always appear in the JSON output.
</div><div class="execute"><div class="execute-title">
<span class="host">pg-primary</span> <b>⇒</b> Get info for the demo cluster
</div><div class="execute-body">
<pre class="execute-body-cmd">sudo -u postgres pgbackrest --stanza=demo --set=20240722-012701F info</pre>
<pre class="execute-body-output">stanza: demo
status: ok
cipher: aes-256-cbc
db (current)
wal archive min/max (12): 000000020000000000000007/000000020000000000000009
full backup: 20240722-012701F
timestamp start/stop: 2024-07-22 01:27:01+00 / 2024-07-22 01:27:04+00
wal start/stop: 000000020000000000000008 / 000000020000000000000009
lsn start/stop: 0/8000028 / 0/9000050
database size: 23.5MB, database backup size: 23.5MB
repo1: backup size: 2.8MB
database list: postgres (13396)</pre>
<pre class="execute-body-output-highlight"> annotation(s)</pre>
<pre class="execute-body-output"> key: value
source: demo backup</pre>
</div></div><div class="section-body-text">
Annotations included with the <span class="cmd">backup</span> command can be added, modified, or removed afterwards using the <span class="cmd">annotate</span> command.
</div><div class="execute"><div class="execute-title">
<span class="host">pg-primary</span> <b>⇒</b> Change backup annotations
</div><div class="execute-body">
<pre class="execute-body-cmd">sudo -u postgres pgbackrest --stanza=demo --set=20240722-012701F \
--annotation=key= --annotation=new_key=new_value annotate</pre>
<pre class="execute-body-cmd">sudo -u postgres pgbackrest --stanza=demo --set=20240722-012701F info</pre>
<pre class="execute-body-output">stanza: demo
status: ok
cipher: aes-256-cbc
db (current)
wal archive min/max (12): 000000020000000000000007/000000020000000000000009
full backup: 20240722-012701F
timestamp start/stop: 2024-07-22 01:27:01+00 / 2024-07-22 01:27:04+00
wal start/stop: 000000020000000000000008 / 000000020000000000000009
lsn start/stop: 0/8000028 / 0/9000050
database size: 23.5MB, database backup size: 23.5MB
repo1: backup size: 2.8MB
database list: postgres (13396)</pre>
<pre class="execute-body-output-highlight"> annotation(s)</pre>
<pre class="execute-body-output"> new_key: new_value
source: demo backup</pre>
</div></div></div></div></div></div><div class="section1"><a id="retention"></a><div class="section1-header"><div class="section1-number"></div><div class="section1-title">
Retention
</div></div><div class="section-body"><div class="section-body-text">
Generally it is best to retain as many backups as possible to provide a greater window for <a href="#pitr">Point-in-Time Recovery</a>, but practical concerns such as disk space must also be considered. Retention options remove older backups once they are no longer needed.
</div><div class="section-body-text">
<span class="backrest">pgBackRest</span> does full backup rotation based on the retention type which can be a count or a time period. When a count is specified, then expiration is not concerned with when the backups were created but with how many must be retained. Differential and Incremental backups are count-based but will always be expired when the backup they depend on is expired. See sections <a href="#retention/full">Full Backup Retention</a> and <a href="#retention/diff">Differential Backup Retention</a> for details and examples. Archived WAL is retained by default for backups that have not expired, however, although not recommended, this schedule can be modified per repository with the retention-archive options. See section <a href="#retention/archive">Archive Retention</a> for details and examples.<br/>
<br/>
The <span class="cmd">expire</span> command is run automatically after each successful backup and can also be run by the user. When run by the user, expiration will occur as defined by the retention settings for each configured repository. If the <span class="br-option">--repo</span> option is provided, expiration will occur only on the specified repository. Expiration can also be limited by the user to a specific backup set with the <span class="br-option">--set</span> option and, unless the <span class="br-option">--repo</span> option is specified, all repositories will be searched and any matching the set criteria will be expired. It should be noted that the archive retention schedule will be checked and performed any time the <span class="cmd">expire</span> command is run.
</div><div class="section2"><a id="retention/full"></a><div class="section2-header"><div class="section2-number"></div><div class="section2-title">
Full Backup Retention
</div></div><div class="section-body"><div class="section-body-text">
The <span class="br-option">repo1-retention-full-type</span> determines how the option <span class="br-option">repo1-retention-full</span> is interpreted; either as the count of full backups to be retained or how many days to retain full backups. New backups must be completed before expiration will occur — that means if <span class="br-setting">repo1-retention-full-type=count</span> and <span class="br-setting">repo1-retention-full=2</span> then there will be three full backups stored before the oldest one is expired, or if <span class="br-setting">repo1-retention-full-type=time</span> and <span class="br-setting">repo1-retention-full=20</span> then there must be one full backup that is at least 20 days old before expiration can occur.
</div><div class="config"><div class="config-title">
<span class="host">pg-primary</span>:<span class="file">/etc/pgbackrest/pgbackrest.conf</span> <b>⇒</b> Configure <span class="br-option">repo1-retention-full</span>
</div><div class="config-body"><div class="config-body-output">
[demo]<br/>
pg1-path=/var/lib/pgsql/12/data<br/>
<br/>
[global]<br/>
repo1-block=y<br/>
repo1-bundle=y<br/>
repo1-cipher-pass=zWaf6XtpjIVZC5444yXB+cgFDFl7MxGlgkZSaoPvTGirhPygu4jOKOXf9LO4vjfO<br/>
repo1-cipher-type=aes-256-cbc<br/>
repo1-path=/var/lib/pgbackrest<br/>
repo1-retention-full=2<br/>
start-fast=y<br/>
<br/>
[global:archive-push]<br/>
compress-level=3
</div></div></div><div class="section-body-text">
Backup <span class="br-setting">repo1-retention-full=2</span> but currently there is only one full backup so the next full backup to run will not expire any full backups.
</div><div class="execute"><div class="execute-title">
<span class="host">pg-primary</span> <b>⇒</b> Perform a full backup
</div><div class="execute-body">
<pre class="execute-body-cmd">sudo -u postgres pgbackrest --stanza=demo --type=full \
--log-level-console=detail backup</pre>
<pre class="execute-body-output"> [filtered 993 lines of output]
P00 INFO: repo1: remove expired backup 20240722-012658F
P00 DETAIL: repo1: 12-1 archive retention on backup 20240722-012701F, start = 000000020000000000000008</pre>
<pre class="execute-body-output-highlight">P00 INFO: repo1: 12-1 remove archive, start = 000000020000000000000007, stop = 000000020000000000000007</pre>
<pre class="execute-body-output">P00 INFO: expire command end: completed successfully</pre>
</div></div><div class="section-body-text">
Archive <i>is</i> expired because WAL segments were generated before the oldest backup. These are not useful for recovery — only WAL segments generated after a backup can be used to recover that backup.
</div><div class="execute"><div class="execute-title">
<span class="host">pg-primary</span> <b>⇒</b> Perform a full backup
</div><div class="execute-body">
<pre class="execute-body-cmd">sudo -u postgres pgbackrest --stanza=demo --type=full \
--log-level-console=info backup</pre>
<pre class="execute-body-output"> [filtered 11 lines of output]
P00 INFO: repo1: expire full backup 20240722-012701F
P00 INFO: repo1: remove expired backup 20240722-012701F</pre>
<pre class="execute-body-output-highlight">P00 INFO: repo1: 12-1 remove archive, start = 000000020000000000000008, stop = 000000020000000000000009</pre>
<pre class="execute-body-output">P00 INFO: expire command end: completed successfully</pre>
</div></div><div class="section-body-text">
The <span class="id">20240722-012645F</span> full backup is expired and archive retention is based on the <span class="id">20240722-012706F</span> which is now the oldest full backup.
</div></div></div><div class="section2"><a id="retention/diff"></a><div class="section2-header"><div class="section2-number"></div><div class="section2-title">
Differential Backup Retention
</div></div><div class="section-body"><div class="section-body-text">
Set <span class="br-option">repo1-retention-diff</span> to the number of differential backups required. Differentials only rely on the prior full backup so it is possible to create a <q>rolling</q> set of differentials for the last day or more. This allows quick restores to recent points-in-time but reduces overall space consumption.
</div><div class="config"><div class="config-title">
<span class="host">pg-primary</span>:<span class="file">/etc/pgbackrest/pgbackrest.conf</span> <b>⇒</b> Configure <span class="br-option">repo1-retention-diff</span>
</div><div class="config-body"><div class="config-body-output">
[demo]<br/>
pg1-path=/var/lib/pgsql/12/data<br/>
<br/>
[global]<br/>
repo1-block=y<br/>
repo1-bundle=y<br/>
repo1-cipher-pass=zWaf6XtpjIVZC5444yXB+cgFDFl7MxGlgkZSaoPvTGirhPygu4jOKOXf9LO4vjfO<br/>
repo1-cipher-type=aes-256-cbc<br/>
repo1-path=/var/lib/pgbackrest<br/>
repo1-retention-diff=1<br/>
repo1-retention-full=2<br/>
start-fast=y<br/>
<br/>
[global:archive-push]<br/>
compress-level=3
</div></div></div><div class="section-body-text">
Backup <span class="br-setting">repo1-retention-diff=1</span> so two differentials will need to be performed before one is expired. An incremental backup is added to demonstrate incremental expiration. Incremental backups cannot be expired independently — they are always expired with their related full or differential backup.
</div><div class="execute"><div class="execute-title">
<span class="host">pg-primary</span> <b>⇒</b> Perform differential and incremental backups
</div><div class="execute-body">
<pre class="execute-body-cmd">sudo -u postgres pgbackrest --stanza=demo --type=diff backup</pre>
<pre class="execute-body-cmd">sudo -u postgres pgbackrest --stanza=demo --type=incr backup</pre>
</div></div><div class="section-body-text">
Now performing a differential backup will expire the previous differential and incremental backups leaving only one differential backup.
</div><div class="execute"><div class="execute-title">
<span class="host">pg-primary</span> <b>⇒</b> Perform a differential backup
</div><div class="execute-body">
<pre class="execute-body-cmd">sudo -u postgres pgbackrest --stanza=demo --type=diff \
--log-level-console=info backup</pre>
<pre class="execute-body-output"> [filtered 10 lines of output]
P00 INFO: backup command end: completed successfully
P00 INFO: expire command begin 2.53: --exec-id=2302-7dc522ec --log-level-console=info --no-log-timestamp --repo1-cipher-pass=<redacted> --repo1-cipher-type=aes-256-cbc --repo1-path=/var/lib/pgbackrest --repo1-retention-diff=1 --repo1-retention-full=2 --stanza=demo</pre>
<pre class="execute-body-output-highlight">P00 INFO: repo1: expire diff backup set 20240722-012709F_20240722-012712D, 20240722-012709F_20240722-012714I</pre>
<pre class="execute-body-output">P00 INFO: repo1: remove expired backup 20240722-012709F_20240722-012714I
P00 INFO: repo1: remove expired backup 20240722-012709F_20240722-012712D
P00 INFO: expire command end: completed successfully</pre>
</div></div></div></div><div class="section2"><a id="retention/archive"></a><div class="section2-header"><div class="section2-number"></div><div class="section2-title">
Archive Retention
</div></div><div class="section-body"><div class="section-body-text">
Although <span class="backrest">pgBackRest</span> automatically removes archived WAL segments when expiring backups (the default expires WAL for full backups based on the <span class="br-option">repo1-retention-full</span> option), it may be useful to expire archive more aggressively to save disk space. Note that full backups are treated as differential backups for the purpose of differential archive retention.
</div><div class="section-body-text">
Expiring archive will never remove WAL segments that are required to make a backup consistent. However, since Point-in-Time-Recovery (PITR) only works on a continuous WAL stream, care should be taken when aggressively expiring archive outside of the normal backup expiration process. To determine what will be expired without actually expiring anything, the <span class="br-option">dry-run</span> option can be provided on the command line with the <span class="cmd">expire</span> command.
</div><div class="config"><div class="config-title">
<span class="host">pg-primary</span>:<span class="file">/etc/pgbackrest/pgbackrest.conf</span> <b>⇒</b> Configure <span class="br-option">repo1-retention-diff</span>
</div><div class="config-body"><div class="config-body-output">
[demo]<br/>
pg1-path=/var/lib/pgsql/12/data<br/>
<br/>
[global]<br/>
repo1-block=y<br/>