v3.0.8

Major features of this stable release are:

• Infrastructure & CI

  • Switched tests and workflows to ubuntu-latest due to 20.04 deprecation.
  • Updated Go version to 1.24 and refreshed linters (golangci-lint v2.0+).
  • Multiple dependency upgrades (Azure SDK, OAuth2, crypto, JWT, gRPC, pgx, compress, etc.).

• PostgreSQL / Greenplum / Cloudberry

  • Added DIRECT-IO Reader for PostgreSQL.
  • New commands for continuous recovery in GPDB/CBDB.
  • Fixes for WAL page size, block size, checksums, and Cloudberry test stability.
  • Support for xid64 patch page layout.

• Redis

  • Enhanced support for sharded Redis backups/restores: slots info, hostname/config parsing, validation fixes.

• MongoDB

  • Introduced partial restore and partial oplog replay.
  • Improved metadata handling and collection in separate goroutines.
  • Dropped tests for deprecated Mongo versions.

• MySQL

  • Added binlog-list command and reconnection to binlog server.
  • Expanded unit tests for metadata fetch/set.

• Storage Backends

  • New implementation for Alibaba Cloud OSS with multipart upload support.
  • Fixes for S3 client headers, batch deletion, SSE-C bug, and versioning behavior.
  • Added S3_DELETE_BATCH_SIZE and none compression method.

• Exporter & Monitoring

  • Created Prometheus exporter with improved metrics (backup size, reduced binary size, better scrape intervals).
  • Performance optimizations in WAL-G exporter.

• Other Fixes

  • Improved error handling (e.g., corrupted WAL metadata, connection refused).
  • Added retries for restore point creation.
  • Various test workflow improvements and cleanup.

What's Changed

• Refactor: Move CheckArchiveCommand from "connect.go" to queryRunner by @boosterKRD in wal-g#1882
• CI: switch unit tests to ubuntu-latest by @chipitsine in wal-g#1965
• Linters update to v2.0 by @debebantur in wal-g#1955
• Add config option WALG_FORCE_WAL_DELTA by @fanfuxiaoran in wal-g#1940
• Switch to container due to Ubuntu 20.04 deprecation on GitHub (fix the worflow release.yml) by @boosterKRD in wal-g#1970
• switch docker tests to ubuntu-latest by @chipitsine in wal-g#1973
• Disable WALG_FORCE_WAL_DELTA in tests by @fanfuxiaoran in wal-g#1975
• update azure sdk by @serprex in wal-g#1982
• Disable HTTP keepalive in endpoint requests by @EinKrebs in wal-g#1976
• golangci-lint 2.1 by @serprex in wal-g#1981
• add slots info for sharded Redis restore by @kronwerk in wal-g#1989
• some fixes for slot collection by @kronwerk in wal-g#1992
• fix Redis slots data paths by @kronwerk in wal-g#1994
• Added retries for restore point creation by @debebantur in wal-g#2000
• Added PGAPPNAME to allowed settings by @debebantur in wal-g#2002
• wal-verify flags with specified lsn and timeline to check by @instahipsta14 in wal-g#2001
• Fix failed restore gp cbdb by @Mulily0513 in wal-g#1974
• DIRECT-IO Reader for PostgreSQL by @vadv in wal-g#2003
• add slots info for sharded rdb backups also by @kronwerk in wal-g#2007
• Fix WalPageSize and BlockSize for gp and cloudberry by @fanfuxiaoran in wal-g#1968
• drop 4.2 and 5.0 mongo_features tests due to deprecated versions by @NotAmigo in wal-g#2009
• add partially restore to mongodb by @NotAmigo in wal-g#1995
• add sharded redis hostname parsing options by @kronwerk in wal-g#2014
• add redis sharded config parsing options by @kronwerk in wal-g#2015
• fix slots validation for redis sharded backups by @kronwerk in wal-g#2016
• update golang.org/x/oauth2 to v0.27.0 by @chipitsine in wal-g#2017
• Add HandleDetailedBackupList table output unit-tests by @BasedDepartment1 in wal-g#2013
• etcd: Delete target backup by @diPhantxm in wal-g#2018
• Update go version to 1.24 by @ostinru in wal-g#2025
• feat: add alicloud OSS storage implementation by @imrenagi in wal-g#2022
• Move copy composer file upload logic to its own method to create a different scope and avoid race condition by @dkhorasanizadeh in wal-g#2020
• Fix the issue where a corrupted WAL metadata file caused failure in pushing WALs by @dkhorasanizadeh in wal-g#2021
• fix: remove warning when alicloud oss valid configurations are present by @imrenagi in wal-g#2028
• Fix greenplum_path.sh usage for Cloudberry by @ostinru in wal-g#2029
• Fix WALG_FAILOVER_STORAGES_CHECK unknown setting warning by @de1ayer in wal-g#2019
• Update partial restore for mongodb by @NotAmigo in wal-g#2026
• tests: rebuild docker image for pg_storage_ssh_* test as well by @chipitsine in wal-g#2030
• Fix GPDB/CBDB/etcd build by @ostinru in wal-g#2033
• Feat: Add new commands to support gpdb/cbdb continuous recovery by @Mulily0513 in wal-g#1998
• remove debug prints by @NotAmigo in wal-g#2032
• Check for pageHeader.isNew before pageHeader.isValid by @bonnefoa in wal-g#2037
• build & release: pick only one tag if there are multiple by @chipitsine in wal-g#2006
• tests: mute ubuntu 20.04 EOL on mariadb setup by @chipitsine in wal-g#2039
• fix: HTTP-headers sent by S3 client by @ostinru in wal-g#2036
• Support the xid64 patch's page layout. by @e-voro in wal-g#2040
• feat: use multipart upload for alicloud oss by @imrenagi in wal-g#2043
• fix cross building of sql server by @chipitsine in wal-g#2046
• Fix cloudberry test by pinning Cloudberry 2.0 as stable version we support by @ostinru in wal-g#2045
• Bump actions/checkout from 4 to 5 by @dependabot[bot] in wal-g#2048
• CI: add sql server cross build by @chipitsine in wal-g#2047
• Add binlog-list command for mysql by @zkramorev in wal-g#2051
• download MongoDB backup metadata only if at least one of white or black lists are specified by @NotAmigo in wal-g#2053
• Added gcflags to enable debugging for build by @debebantur in wal-g#2052
• Bump gopkg.in/yaml.v3 from 3.0.0 to 3.0.1 in /internal/tools by @dependabot[bot] in wal-g#2054
• Bump actions/checkout from 4 to 5 by @dependabot[bot] in wal-g#2055
• add flag to count journal info by @kronwerk in wal-g#2049
• Bump actions/setup-go from 5 to 6 by @dependabot[bot] in wal-g#2059
• do not delete key from tarfileset if file has been already found by @NotAmigo in wal-g#2057
• Use cast to convert config values by @NikkoFox in wal-g#2034
• Add 'connection refused' to ShouldRetry by @ostinru in wal-g#2061
• Separate metadata MongoDB operations to goroutine && add mechanism of metadata collectiong by @NotAmigo in wal-g#2060
• fix: defer to delete temporary directory by @imrenagi in wal-g#2066
• Greenplum follow-primary: use absolute path to pg_ctl by @ostinru in wal-g#2063
• Create prometheus exporter by @dementii-priadko in wal-g#2064
• Greenplum follow-primary: print missing WAL file names by @ostinru in wal-g#2070
• tests: do not use MONGO_MAJOR anymore by @chipitsine in wal-g#2058
• tests: migrate from "bitnami" to "bitnamilegacy" by @chipitsine in wal-g#2076
• cleanup: do not install "pinentry-qt" to pg test container by @chipitsine in wal-g#2078
• do not use deprecated "-d" flag in "go get" by @chipitsine in wal-g#2079
• PostgreSQL & Greenplum: Reduce memory usage by @ostinru in wal-g#2056
• sequential test workflow: allow manual execution in forks by @chipitsine in wal-g#2077
• prepare test scripts for Postgresql > 10 by @chipitsine in wal-g#2080
• added GODEBUG to known settings by @debebantur in wal-g#2082
• Partial oplog replay by @NotAmigo in wal-g#2069
• preparation for Postgres > 10 by @chipitsine in wal-g#2084
• tests: run workflows on any branch by @chipitsine in wal-g#2083
• tests: drop /tmp/scripts/wait_while_pg_not_ready.sh wherever possible by @chipitsine in wal-g#2085
• Reduce walg-exporter binary siz...

v3.0.7

Major features of this stable release are:

• From Greenplum's perspective, the following updates are beneficial: Commit 92e3995 introduces a --force-delete flag to the delete command. This enhancement enables users to effectively remove backups, even in cases where earlier deletion attempts were unsuccessful.

• Commit 094bcb4 for Cloudberry: Added support for recognizing 'Apache Cloudberry' in the version() command, facilitating users in working with Cloudberry from the trunk branch.

• Commit 44e4386 enhances the supply chain, resolving issues flagged by the Trivy security scanner.

What's Changed

• tests: remove few "sleep" by @chipitsine in wal-g#1883
• add unit test to /wal-g/internal/databases/greenplum/backup_push_hand… by @hairy-snake in wal-g#1890
• tests: follow redirects when building MariaDB image by @chipitsine in wal-g#1889
• Bump golangci/golangci-lint-action from 6.1.1 to 6.2.0 by @dependabot in wal-g#1891
• Add JournalInfo deletion logic by @noname0443 in wal-g#1874
• Refactor tests for postgres cleaner using gomock by @kozlov-ma in wal-g#1892
• Fixing issue by @M1ha-Shvn in wal-g#1842
• storages (s3): Add AWS_DUAL_STACK configuration by @sundbry in wal-g#1691
• Update pgx to v5 by @debebantur in wal-g#1893
• docker: use minio from docker hub (every time) by @chipitsine in wal-g#1895
• add user to Redis connection by @kronwerk in wal-g#1894
• replace redis aof backup pin method by @kronwerk in wal-g#1899
• remove folder cleanup by @kronwerk in wal-g#1901
• add user data into etcd sentinel by @diPhantxm in wal-g#1902
• Greenplum: add --force-delete flag to delete command by @ostinru in wal-g#1903
• Added working with S3 versions 2 by @debebantur in wal-g#1900
• pg_tests: try to prevent race condition on background start by @chipitsine in wal-g#1896
• Add unit tests for internal/databases/redis.HandleDetailedBackupList by @kozlov-ma in wal-g#1876
• Bump golangci/golangci-lint-action from 6.2.0 to 6.3.1 by @dependabot in wal-g#1907
• Reuse multistorage for daemon mode by @debebantur in wal-g#1908
• MongoDB oplog replay: skip startIndexBuild and abortIndexBuild ops + some refactoring by @VGoshev in wal-g#1911
• Better gp multistorage by @munakoiso in wal-g#1898
• Add @kronwerk to MongoDB codeowners by @VGoshev in wal-g#1914
• MongoDB Oplog replay: fix skipping some ops + fix issue in indexSpecFromCommitIndexBuilds by @VGoshev in wal-g#1913
• remove one more sleep from tests by @chipitsine in wal-g#1904
• MySQL: apply xtrabackup deltas by @ostinru in wal-g#1831
• fix assume-role by @dbarrosop in wal-g#1906
• Fix connection break by @debebantur in wal-g#1917
• Cloudberry: accept 'Apache Cloudbery' in version() by @ostinru in wal-g#1919
• Wal-g write an error to logs when there are no backups in a S3 storage by @noname0443 in wal-g#1918
• Remove many recursive queries in metadata collection by @debebantur in wal-g#1920
• add max db number to walg sentinel by @kronwerk in wal-g#1921
• differ 0 and missing data by @kronwerk in wal-g#1922
• Bump uraimo/run-on-arch-action from 2 to 3 by @dependabot in wal-g#1925
• Fix index out of range in wal-g xb extract by @ostinru in wal-g#1923
• Add wal-g st get-stream command to fetch backup as a single stream by @ostinru in wal-g#1924
• fix typo by @munakoiso in wal-g#1926
• MySQL: fix how --inplace handles delta files by @ostinru in wal-g#1928
• Do not run tests twice when PR submitted by @ostinru in wal-g#1931
• Remove redundant debug by @munakoiso in wal-g#1935
• Fix Cloudberry Tests - follow upstream changes by @ostinru in wal-g#1933
• Fix postgresql tests by @debebantur in wal-g#1934
• migrate to new issue templates by @boosterKRD in wal-g#1943
• Allow restore between different patches of the same redis minor version by @zykovd in wal-g#1942
• Added deduplication of pgdata in path by @debebantur in wal-g#1946
• Reduce logging for partial database restore [pg/gp] by @ostinru in wal-g#1947
• move pgconn and pgproto used into pgx by @fepape-ionos in wal-g#1941
• golangci-lint 1.64 by @serprex in wal-g#1910
• refactoring oplog replay by @Apelsin234 in wal-g#1950
• releases: add ubuntu22 for aarch64 by @chipitsine in wal-g#1956
• Fix catchup issuing premature stop by @x4m in wal-g#1957
• Prefetched wal is expected in default directory by @debebantur in wal-g#1954

New Contributors

• @hairy-snake made their first contribution in wal-g#1890
• @kozlov-ma made their first contribution in wal-g#1892
• @M1ha-Shvn made their first contribution in wal-g#1842
• @sundbry made their first contribution in wal-g#1691
• @diPhantxm made their first contribution in wal-g#1902
• @dbarrosop made their first contribution in wal-g#1906

Full Changelog: wal-g/wal-g@v3.0.5...v3.0.7

v3.0.6

Synced from wal-g/wal-g v3.0.6

v3.0.5

First stable release in 2025. Major features of this stable release are:

• Cloudberry database support
• MongoDB 8.0 support + restore for sharded cluster with PiTR
• PostgreSQL - fix wal-g copy

Additionally this release contains a bunch of bugfixes and documentation updates. For MySQL we fixed regression (all versions since v3.0.0 affected) that leads to broken backups to be marked as good ones. We strongly recommend MySQL users to upgrade.

What's Changed

• Add option to disable 100 continue in s3 by @fepape-ionos in wal-g#1829
• pg_checksum_verification_fix by @boosterKRD in wal-g#1828
• fix: don't ignore libsodiumKeyPath by @fepape-ionos in wal-g#1832
• CI: drop ubuntu-18.04 from release pipeline by @chipitsine in wal-g#1825
• fix data race on accessing bool var from different goroutines by @kronwerk in wal-g#1835
• MongoDB: Support restore of sharded cluster with PiTR by @Apelsin234 in wal-g#1797
• MongoDB: don't skip error if file was deleted while make backup by @Apelsin234 in wal-g#1838
• CI: cleanup build process a bit by @chipitsine in wal-g#1834
• add ubuntu-22, ubuntu-24 to release matrix by @chipitsine in wal-g#1841
• fix IsDirectoryEmpty error by @NotAmigo in wal-g#1836
• feat(file): add sync mutex to TarFileSet by @SoulKyu in wal-g#1843
• bump golang version in docker tests by @ostinru in wal-g#1840
• update brotli submodule by @chipitsine in wal-g#1791
• fix: correct LSN format string in delta backup logging by @robertmu in wal-g#1848
• Unit-tests to Upload with mocks by @Amyton0 in wal-g#1837
• Changed error handling while collecting tables data by @debebantur in wal-g#1849
• MySQL: Fix regression - broken backup considered as good one by @ostinru in wal-g#1852
• postgres: refactor backup to skip non-physical storage objects (wal-g#1850) by @robertmu in wal-g#1851
• release: unify amd64 <--> arm64 names by @chipitsine in wal-g#1854
• Fix wal-g copy for pg by @munakoiso in wal-g#1856
• MySQL: don't add --apply-log-only to the latest stage of backup preparation. by @ostinru in wal-g#1857
• MongoDB: Add 8.0 support + bump other versions minors by @VGoshev in wal-g#1845
• Fix a grammar error in the PR template by @dligr in wal-g#1865
• wal-g copy for greenplum by @munakoiso in wal-g#1862
• Add logging to file by @EinKrebs in wal-g#1866
• fix(postgres): resolve race condition in pg_wal_restore_test by @robertmu in wal-g#1868
• fix readthedocs build definition by @chipitsine in wal-g#1847
• bump libsodium to 1.0.20 by @chipitsine in wal-g#1870
• do not run scheduled tests in forks by @chipitsine in wal-g#1858
• make mongo docker compose tests more stable by @chipitsine in wal-g#1789
• fix(PostgreSQL): exclude transaction state files from page file detec… by @robertmu in wal-g#1864
• docs: update typo in MySQL.md by @mr-ssd in wal-g#1877
• tests: remove few "sleep" by @chipitsine in wal-g#1878
• tests: use $(nproc) instead of hardcoded "8" by @chipitsine in wal-g#1881
• Fix SIGSEGV errors of wal-g delete by @ostinru in wal-g#1879
• WALG-1873. MySQL > use datadir as incremental dir in tests by @ostinru in wal-g#1880
• Greenplum multistorage support by @munakoiso in wal-g#1875
• Fixed backward compatibility with files metadata by @debebantur in wal-g#1886
• Fix Postgres 12+/Cloudberry concurrent backups. by @ostinru in wal-g#1813

New Contributors

• @fepape-ionos made their first contribution in wal-g#1829
• @SoulKyu made their first contribution in wal-g#1843
• @robertmu made their first contribution in wal-g#1848
• @Amyton0 made their first contribution in wal-g#1837
• @dligr made their first contribution in wal-g#1865
• @mr-ssd made their first contribution in wal-g#1877

Full Changelog: wal-g/wal-g@v3.0.4...v3.0.5