Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Provide commands to check WAL archive destination is usable #443

Merged
merged 3 commits into from
Nov 16, 2021
Merged

Provide commands to check WAL archive destination is usable #443

merged 3 commits into from
Nov 16, 2021

Commits on Nov 16, 2021

  1. Provide commands to check WAL archive destination is usable 

    Adds commands to barman and barman cloud which check that a barman server
or cloud location is safe to use as an archive destination for a new
PostgreSQL server.

A location is considered safe if either:

1. There are no WAL files at all in the archive.
2. All existing WAL files belong to an older timeline than that
   specified by the --timeline argument.

A file is considered a WAL file if it passes the `is_any_xlog_file` check
in `barman/xlog.py` so this applies to WAL files, history files, partial
WAL files and backup labels.

The commands added are:

* barman check-wal-archive
* barman-cloud-check-wal-archive

The motivation for this patch is to provide a way that external
orchestration tools can validate the WAL archive destination is safe for
a newly provisioned PostgreSQL cluster, given such a cluster may use the
exact same name as an old cluster.

In such scenarios, any WAL files on the same or higher timeline as the
WALs being written by the new cluster will cause any attempt to restore
from a backup to fail.

Reasons why external orchestration tooling may re-use the same cluster
name and archive destination include (but are not limited to):

* A new cluster is created via initdb with the same name as the old one.
  The sysid will be different but this does not affect the archive
  destination so any archived WALs relating to the older cluster will be
  present in the same location.
* A cluster is restored from a base backup and uses the same name as the
  old cluster. The cluster has the same sysid and starts with a segment
  ID > 1 and timeline > 1. The same archive destination used by the old
  cluster will be used for the restored cluster.
* A new cluster is started which happens to re-use the same name and
  archive destination.

All of these cases lead to the situation where WAL archiving and backup
is functioning normally *but* any attempts to restore from those backups
will fail. This is dangerous for anyone relying on the databases managed
by external orchestration/automation.

The commands provided by this patch do not solve the problem alone
because neither Barman nor PostgreSQL have the necessary context. The
commands can, however, be added to external automation in order to
catch archive safety issues at the provisioning stage.

Closes #432
    @mikewallace1979
    mikewallace1979 committed Nov 16, 2021
    Configuration menu
    Copy the full SHA
    262b319 View commit details
    Browse the repository at this point in the history

  2. Update docs for check-wal-archive commands

    @mikewallace1979
    mikewallace1979 committed Nov 16, 2021
    Configuration menu
    Copy the full SHA
    3003446 View commit details
    Browse the repository at this point in the history

  3. Remove friendly argparse import errors 

    We only support python versions which ship with argparse now so it is no
longer necessary to catch ImportError and print a friendly error when
importing argparse.
    @mikewallace1979
    mikewallace1979 committed Nov 16, 2021
    Configuration menu
    Copy the full SHA
    875fb4e View commit details
    Browse the repository at this point in the history