Assets 2

Bug Fixes:

  • Fix missing missing URI encoding in S3 driver. (Reported by Dan Farrell.)
  • Fix incorrect error message for duplicate options in configuration files. (Reported by Jesper St John.)
  • Fix incorrectly reported error return in info logging. A return code of 1 from the archive-get was being logged as an error message at info level but otherwise worked correctly.

Features:

  • Add checksum delta for incremental backups which uses checksums rather than timestamps to determine if files have changed. (Contributed by Cynthia Shang.)
  • PostgreSQL 11 support, including configurable WAL segment size.

Improvements:

  • Ignore all files in a linked tablespace directory except the subdirectory for the current version of PostgreSQL. Previously an error would be generated if other files were present and not owned by the PostgreSQL user.
  • Improve info command to display the stanza cipher type. (Contributed by Cynthia Shang. Suggested by Douglas J Hunley.)
  • Improve support for special characters in filenames.
  • Allow delta option to be specified in the pgBackRest configuration file. (Contributed by Cynthia Shang.)
Assets 2

Bug Fixes:

  • Fix issue where relative links in $PGDATA could be stored in the backup with the wrong path. This issue did not affect absolute links and relative tablespace links were caught by other checks. (Reported by Cynthia Shang.)
  • Remove incompletely implemented online option from the check command. Offline operation runs counter to the purpose of this command, which is to check if archiving and backups are working correctly. (Reported by Jason O'Donnell.)
  • Fix issue where errors raised in C were not logged when called from Perl. pgBackRest properly terminated with the correct error code but lacked an error message to aid in debugging. (Reported by Douglas J Hunley.)
  • Fix issue when a boolean option (e.g. delta) was specified more than once. (Reported by Yogesh Sharma.)

Features:

  • Allow any option to be set in an environment variable. This includes options that previously could only be specified on the command line, e.g. stanza, and secret options that could not be specified on the command-line, e.g. repo1-s3-key-secret.
  • Exclude temporary and unlogged relation (table/index) files from backup. Implemented using the same logic as the patches adding this feature to PostgreSQL, 8694cc96 and 920a5e50. Temporary relation exclusion is enabled in PostgreSQL ≥ 9.0. Unlogged relation exclusion is enabled in PostgreSQL ≥ 9.1, where the feature was introduced. (Contributed by Cynthia Shang.)
  • Allow arbitrary directories and/or files to be excluded from a backup. Misuse of this feature can lead to inconsistent backups so read the --exclude documentation carefully before using. (Reviewed by Cynthia Shang.)
  • Add log-subprocess option to allow file logging for local and remote subprocesses.
  • PostgreSQL 11 Beta 3 support.

Improvements:

  • Allow zero-size files in backup manifest to reference a prior manifest regardless of timestamp delta. (Contributed by Cynthia Shang.)
  • Improve asynchronous archive-get/archive-push performance by directly checking status files. (Contributed by Stephen Frost.)
  • Improve error message when a command is missing the stanza option. (Suggested by Sarah Conway.)

@dwsteele dwsteele released this Jul 6, 2018 · 125 commits to master since this release

Assets 2

IMPORTANT NOTE: This release fixes a critical bug in the backup resume feature. All resumed backups prior to this release should be considered inconsistent. A backup will be resumed after a prior backup fails, unless resume=n has been specified. A resumed backup can be identified by checking the backup log for the message "aborted backup of same type exists, will be cleaned to remove invalid files and resumed". If the message exists, do not use this backup or any backup in the same set for a restore and check the restore logs to see if a resumed backup was restored. If so, there may be inconsistent data in the cluster.

Bug Fixes:

  • Fix critical bug in resume that resulted in inconsistent backups. A regression in v0.82 removed the timestamp comparison when deciding which files from the aborted backup to keep on resume. See note above for more details. (Reported by David Youatt, Yogesh Sharma, Stephen Frost.)
  • Fix error in selective restore when only one user database exists in the cluster. (Fixed by Cynthia Shang. Reported by Nj Baliyan.)
  • Fix non-compliant ISO-8601 timestamp format in S3 authorization headers. AWS and some gateways were tolerant of space rather than zero-padded hours while others were not. (Fixed by Andrew Schwartz.)

Features:

  • PostgreSQL 11 Beta 2 support.

Improvements:

  • Improve the HTTP client to set content-length to 0 when not specified by the server. S3 (and gateways) always set content-length or transfer-encoding but HTTP 1.1 does not require it and proxies (e.g. HAProxy) may not include either. (Suggested by Adam K. Sumner.)
  • Set search_path = 'pg_catalog' on PostgreSQL connections. (Suggested by Stephen Frost.)
Assets 2

IMPORTANT NOTE: This release fixes a critical bug in the backup resume feature. All resumed backups prior to this release should be considered inconsistent. A backup will be resumed after a prior backup fails, unless resume=n has been specified. A resumed backup can be identified by checking the backup log for the message "aborted backup of same type exists, will be cleaned to remove invalid files and resumed". If the message exists, do not use this backup or any backup in the same set for a restore and check the restore logs to see if a resumed backup was restored. If so, there may be inconsistent data in the cluster.

Bug Fixes:

  • Fix critical bug in resume that resulted in inconsistent backups. A regression in v0.82 removed the timestamp comparison when deciding which files from the aborted backup to keep on resume. See note above for more details. (Reported by David Youatt, Yogesh Sharma, Stephen Frost.)
  • Fix non-compliant ISO-8601 timestamp format in S3 authorization headers. AWS and some gateways were tolerant of space rather than zero-padded hours while others were not. (Fixed by Andrew Schwartz.)
  • Fix directory syncs running recursively when only the specified directory should be synced. (Reported by Craig A. James.)
  • Fix --target-action and --recovery-option options being reported as invalid when restoring with --type=immediate. (Reported by Brad Nicholson.)
  • Fix archive-copy throwing "path not found" error for incr/diff backups. (Reported by yummyliu, Vitaliy Kukharik.)
  • Fix failure in manifest build when two or more files in PGDATA are linked to the same directory. (Reported by Vitaliy Kukharik.)
  • Fix delta restore failing when a linked file was missing.
  • Fix error in selective restore when only one user database exists in the cluster. (Fixed by Cynthia Shang. Reported by Nj Baliyan.)

Improvements:

  • Improve the HTTP client to set content-length to 0 when not specified by the server. S3 (and gateways) always set content-length or transfer-encoding but HTTP 1.1 does not require it and proxies (e.g. HAProxy) may not include either. (Suggested by Adam K. Sumner.)
  • Improve performance of HTTPS client. Buffering now takes the pending bytes on the socket into account (when present) rather than relying entirely on select(). In some instances the final bytes would not be flushed until the connection was closed.
  • Improve S3 delete performance. The constant S3_BATCH_MAX had been replaced with a hard-coded value of 2, probably during testing.
  • Make backup/restore path sync more efficient. Scanning the entire directory can be very expensive if there are a lot of small tables. The backup manifest contains the path list so use it to perform syncs instead of scanning the backup/restore path. Remove recursive path sync functionality since it is no longer used.

@dwsteele dwsteele released this May 23, 2018 · 155 commits to master since this release

Assets 2

Bug Fixes:

  • Fix potential buffer overrun in error message handling. (Reported by Lætitia.)
  • Fix archive write lock being taken for the synchronous archive-get command. (Reported by Uspen.)

Improvements:

  • Embed exported C functions and Perl modules directly into the pgBackRest executable.
  • Use time_t instead of __time_t for better portability. (Suggested by Nick Floersch.)
  • Print total runtime in milliseconds at command end.
Assets 2

Bug Fixes:

  • Fix directory syncs running recursively when only the specified directory should be synced. (Reported by Craig A. James.)
  • Fix archive-copy throwing "path not found" error for incr/diff backups. (Reported by yummyliu, Vitaliy Kukharik.)
  • Fix failure in manifest build when two or more files in PGDATA are linked to the same directory. (Reported by Vitaliy Kukharik.)
  • Fix delta restore failing when a linked file is missing.
  • Fix rendering of key/value and list options in help. (Reported by Clinton Adams.)

Features:

  • Add asynchronous, parallel archive-get. This feature maintains a queue of WAL segments to help reduce latency when PostgreSQL requests a WAL segment with restore_command.
  • Add support for additional pgBackRest configuration files in the directory specified by the --config-include-path option. Add --config-path option for overriding the default base path of the --config and --config-include-path option. (Contributed by Cynthia Shang.)
  • Add repo-s3-token option to allow temporary credentials tokens to be configured. pgBackRest currently has no way to request new credentials so the entire command (e.g. backup, restore) must complete before the credentials expire. (Contributed by Yogesh Sharma.)

Improvements:

  • Update the archive-push-queue-max, manifest-save-threshold, and buffer-size options to accept values in KB, MB, GB, TB, or PB where the multiplier is a power of 1024. (Contributed by Cynthia Shang.)
  • Make backup/restore path sync more efficient. Scanning the entire directory can be very expensive if there are a lot of small tables. The backup manifest contains the path list so use it to perform syncs instead of scanning the backup/restore path.
  • Show command parameters as well as command options in initial info log message.
  • Rename archive-queue-max option to archive-push-queue-max to avoid confusion with the new archive-get-queue-max option. The old option name will continue to be accepted.

@dwsteele dwsteele released this Mar 20, 2018 · 243 commits to master since this release

Assets 2

Bug Fixes:

  • Fix --target-action and --recovery-option options being reported as invalid when restoring with --type=immediate. (Reported by Brad Nicholson.)
  • Immediately error when a secure option (e.g. repo1-s3-key) is passed on the command line. Since pgBackRest would not pass secure options on to sub-processes an obscure error was thrown. The new error is much clearer and provides hints about how to fix the problem. Update command documentation to omit secure options that cannot be specified on the command-line. (Reported by Brad Nicholson.)
  • Fix issue passing --no-config to embedded Perl. (Reported by Ibrahim Edib Kokdemir.)
  • Fix issue where specifying log-level-stderr > warn would cause a local/remote process to error on exit due to output found on stderr when none was expected. The max value for a local/remote process is now error since there is no reason for these processes to emit warnings. (Reported by Clinton Adams.)
  • Fix manifest test in the check command when tablespaces are present. (Fixed by Cynthia Shang. Reported by Thomas Flatley.)

Improvements:

  • Error when multiple arguments are set in the config file for an option that does not accept multiple arguments. (Contributed by Cynthia Shang.)
  • Remove extraneous sudo commands from src/Makefile. (Contributed by Adrian Vondendriesch.)
Assets 2

Features:

  • The archive-push command is now partially coded in C which allows the PostgreSQL archive_command to run significantly faster when processing status messages from the asynchronous archive process. (Reviewed by Cynthia Shang.)

Improvements:

  • Improve check command to verify that the backup manifest can be built. (Contributed by Cynthia Shang.)
  • Improve performance of HTTPS client. Buffering now takes the pending bytes on the socket into account (when present) rather than relying entirely on select(). In some instances the final bytes would not be flushed until the connection was closed.
  • Improve S3 delete performance. The constant S3_BATCH_MAX had been replaced with a hard-coded value of 2, probably during testing.
  • Allow any non-command-line option to be reset to default on the command-line. This allows options in pgbackrest.conf to be reset to default which reduces the need to write new configuration files for specific needs.
  • The C library is now required. This eliminates conditional loading and eases development of new library features.
  • The pgbackrest executable is now a C binary instead of Perl. This allows certain time-critical commands (like async archive-push) to run more quickly.
  • Rename db-* options to pg-* and backup-* options to repo-* to improve consistency. repo-* options are now indexed although currently only one is allowed.

@dwsteele dwsteele released this Feb 1, 2018 · 22 commits to release/1 since this release

Assets 2

Bug Fixes:

  • Fixed inability to restore a single database contained in a tablespace using --db-include. (Fixed by Cynthia Shang.)
  • Ensure latest db-id is selected on when matching archive.info to backup.info. This provides correct matching in the event there are system-id and db-version duplicates (e.g. after reverting a pg_upgrade). (Fixed by Cynthia Shang. Reported by Adam K. Sumner.)
  • Fixed overly chatty error message when reporting an invalid command. (Reported by Jason O'Donnell.)

Features:

  • Add stanza-delete command to cleanup unused stanzas. (Contributed by Cynthia Shang. Suggested by Magnus Hagander.)

Improvements:

  • Improve stanza-create command so that it does not error when the stanza already exists. (Contributed by Cynthia Shang.)

@dwsteele dwsteele released this Dec 20, 2017 · 32 commits to release/1 since this release

Assets 2

Bug Fixes:

  • Fixed an issue that suppressed locality errors for backup and restore. When a backup host is present, backups should only be allowed on the backup host and restores should only be allowed on the database host unless an alternate configuration is created that ignores the remote host. (Reported by Lardière Sébastien.)
  • Fixed an issue where WAL was not expired on PostgreSQL 10. This was caused by a faulty regex that expected all PostgreSQL major versions to be X.X. (Reported by Adam Brusselback.)
  • Fixed an issue where the --no-config option was not passed to child processes. This meant the child processes would still read the local config file and possibly cause unexpected behaviors.
  • Fixed info command to eliminate "db (prior)" output if no backups or archives exist for a prior version of the cluster. (Fixed by Cynthia Shang. Reported by Stephen Frost.)