feat: support using zstd for snapshots, fix postgres:9 snapshot, fixes #7844, fixes #3583

[deviantintegral]

The Issue

• Use zstd instead of gzip for database snapshots #7844
• ddev snapshot restore doesn't work with postgres:9 #3583

Implements zstd compression for database snapshots.
Adds snapshot support for postgres:9

For a 19GB mariadb database, this change:

• Reduces exports from around 6:30 to 0:30 (on my M2 Max Macbook Pro).
• Reduces restores from 2:21 to 1:38.
• The test database I'm using compresses to roughly the same size at 2.9 GB, so disk space isn't any worse.

How This PR Solves The Issue

We still support importing gzip databases, but warn the user to re-export. They could manually convert the backup too, but I think it's better to encourage the user to to follow our tools so we can control the export process.

Manual Testing Instructions

• Run ddev snapshot and ddev snapshot restore. I only have mariadb databases, and tested with the 10.6 image.

Automated Testing Overview

• Do automated tests cover this well enough? I would suspect that functional tests of the import and export command are in place. Let's see if CI passes.

Release/Deployment Notes

None.

[github-actions]

Download the artifacts for this pull request:

• all-ddev-executables.zip
• ddev-macos-amd64.zip
• ddev-macos-arm64.zip
• ddev-linux-arm64.zip
• ddev-linux-amd64.zip
• ddev-windows-amd64.zip
• ddev-windows-amd64-installer.zip
• ddev-windows-arm64.zip
• ddev-windows-arm64-installer.zip

See Testing a PR.

[stasadev]

I will push new Docker images.

[stasadev]

It looks like it's not installed by default everywhere:

https://github.com/ddev/ddev/actions/runs/19301022329/job/55195278110?pr=7845#step:13:4585

"/var/tmp/mariadb_5.5_4af8dc72702d23baf7ef8cd09bb15e45-mariadb_5.5.zst" )]', err='exit status 1', stdout='', stderr='bash: zstdmt: command not found'

[stasadev]

The build is successful for everything, except mariadb:5.5

https://github.com/stasadev/ddev/actions/runs/19308126195/job/55220901864

$ docker run --entrypoint=bash -it --rm ddev/ddev-dbserver-mariadb-5.5:v1.24.10

root@f67a1205a0fb:/# cat /etc/os-release
NAME="Ubuntu"
VERSION="14.04.6 LTS, Trusty Tahr"

root@f67a1205a0fb:/# apt-get update && apt-get install zstd
...
E: Unable to locate package zstd

It was added in Ubuntu 18.04 https://balintreczey.hu/blog/hello-zstd-compressed-debs-in-ubuntu/

[rfay]

If it's difficult to get zstd onto mariadb:5.5, it's OK to disable snapshots for that or to insist on gzipped snapshots.

[stasadev]

I disabled snapshots for mariadb:5.5.

or to insist on gzipped snapshots

We can come back to this later.

[stasadev]

Doesn't work for mysql:5.5

https://github.com/ddev/ddev/actions/runs/19309228694/job/55224771825#step:13:7284

bash: zstdmt: command not found

ddevapp_test.go:1738: could not create snapshot mysql_5.5_1215af4cdcf5b5d6cf117dd60e9bcb01 for mysql:5.5: composeCmd failed to run 'COMPOSE_PROJECT_NAME=ddev-testpkgwordpress docker-compose -f /home/runner/tmp/ddevtest/TestPkgWordpress1832718576/.ddev/.ddev-docker-compose-full.yaml exec -T db bash -c set -eu && ( set -eu -o pipefail; xtrabackup --backup --stream=xbstream --user=root --*** --socket=/var/tmp/mysql.sock  2>/tmp/snapshot_mysql_5.5_1215af4cdcf5b5d6cf117dd60e9bcb01-mysql_5.5.zst.log | zstdmt --quiet > "/var/tmp/mysql_5.5_1215af4cdcf5b5d6cf117dd60e9bcb01-mysql_5.5.zst" )', action='[exec -T db bash -c set -eu && ( set -eu -o pipefail; xtrabackup --backup --stream=xbstream --user=root --*** --socket=/var/tmp/mysql.sock  2>/tmp/snapshot_mysql_5.5_1215af4cdcf5b5d6cf117dd60e9bcb01-mysql_5.5.zst.log | zstdmt --quiet > "/var/tmp/mysql_5.5_1215af4cdcf5b5d6cf117dd60e9bcb01-mysql_5.5.zst" )]', err='exit status 127', stdout='', stderr='bash: zstdmt: command not found' output=, saved approot=~/TestDdevAllDatabases-broken-WDoZV

$ ddev exec -s db zstd --version
*** zstd command line interface 64-bits v1.1.2, by Yann Collet ***

$ ddev exec -s db zstdmt        
bash: zstdmt: command not found
Failed to execute command `zstdmt`: exit status 127

$ ddev exec -s db lsb_release -a
No LSB modules are available.
Distributor ID: Debian
Description:    Debian GNU/Linux 9.9 (stretch)
Release:        9.9
Codename:       stretch

[rfay]

I thought they would be from the same era. Neither needs to support snapshots.

[deviantintegral]

zstdmt is an alias for zstd -T0 I think. Double check and perhaps we can use that instead?

[stasadev]

zstdmt is an alias for zstd -T0 I think. Double check and perhaps we can use that instead?

Is this going to work for tar -I zstdmt ?

[deviantintegral]

It does if you wrap it in single quotes. Pushed a change doing that!

[deviantintegral]

It looks like we have some upstream network issues.

#44 [ddev-webserver-dev-base 10/36] RUN set -x && set -o pipefail && tag=$(curl -L --fail --silent "https://api.github.com/repos/axllent/mailpit/releases/latest" | jq -r .tag_name) && curl --fail -sSL "[https://github.com/axllent/mailpit/releases/download/${tag}/mailpit-linux-amd64.tar.gz](https://github.com/axllent/mailpit/releases/download/$%7Btag%7D/mailpit-linux-amd64.tar.gz)" -o /tmp/mailpit.tar.gz && tar -zx -C /usr/local/bin -f /tmp/mailpit.tar.gz mailpit && rm /tmp/mailpit.tar.gz
#44 0.126 + set -o pipefail
#44 0.127 ++ curl -L --fail --silent https://api.github.com/repos/axllent/mailpit/releases/latest
#44 0.127 ++ jq -r .tag_name
#44 0.250 + tag=v1.27.11
#44 0.250 + curl --fail -sSL https://github.com/axllent/mailpit/releases/download/v1.27.11/mailpit-linux-amd64.tar.gz -o /tmp/mailpit.tar.gz
#44 1.068 + tar -zx -C /usr/local/bin -f /tmp/mailpit.tar.gz mailpit
#44 1.254 + rm /tmp/mailpit.tar.gz
#44 DONE 1.3s

#45 [ddev-webserver-dev-base 11/36] RUN curl -sSL --fail --output /usr/local/bin/phive "https://phar.io/releases/phive.phar" && chmod 777 /usr/local/bin/phive
#45 2.762 curl: (22) The requested URL returned error: 503
#45 ERROR: process "/bin/bash -eu -o pipefail -c curl -sSL --fail --output /usr/local/bin/phive \"[https://phar.io/releases/phive.phar\](https://phar.io/releases/phive.phar/)" && chmod 777 /usr/local/bin/phive" did not complete successfully: exit code: 22
------
 > [ddev-webserver-dev-base 11/36] RUN curl -sSL --fail --output /usr/local/bin/phive "https://phar.io/releases/phive.phar" && chmod 777 /usr/local/bin/phive:
2.762 curl: (22) The requested URL returned error: 503

[rfay]

Getting pretty familiar with these! https://www.githubstatus.com/

[deviantintegral]

In https://github.com/ddev/ddev/actions/runs/19575538227/job/56059696814?pr=7845:

#25 0.134 mv: cannot stat 'frankenphp': No such file or directory

It's not clear to me how this is related to the changes in this PR. Any pointers?

[stasadev]

It's not clear to me how this is related to the changes in this PR. Any pointers?

It looks like it's a change upstream, not related to this PR, I'll take a look.

[tyler36]

Tested with Postgres:12 and Postgres:16 and it worked well.
I was using small database but it was consistently a few seconds faster then 1.24.10 snapshots.

• ✅ Backup database
• ✅ Restore zst database
• ✅ Restore gz database

I appreciate the following line:

This snapshot was created with gzip. It's recommended to create a new snapshot using zstd for better performance.

It makes me wonder, "How do I change to zstd?".

Happy to "approve" once the upstream issue (?) is resolved.

[stasadev]

Rebased, dropped merge commit, resolved conflicts, pushed images:

ddev utility download-images --all

[stasadev]

zstdmt -> zstd -T0 didn't change anything 😭, this flag is not available in the old zstd.

---

https://github.com/ddev/ddev/actions/runs/19666937210/job/56326105327?pr=7845#step:13:7328

Creating database snapshot mysql_5.5_aa90952da858a2a4c16f303ec9ff1c9f
Incorrect parameters
Usage :
      zstd [args] [FILE(s)] [-o file]

FILE    : a filename
          with no FILE, or when FILE is - , read standard input
Arguments :
 -#     : # compression level (1-19, default:3) 
 -d     : decompression 
 -D file: use `file` as Dictionary 
 -o file: result stored into `file` (only if 1 input file) 
 -f     : overwrite output without prompting 
--rm    : remove source file(s) after successful de/compression 
 -k     : preserve source file(s) (default) 
 -h/-H  : display help/long help and exit

[deviantintegral]

@tyler36 re:

It makes me wonder, "How do I change to zstd?".

Any new snapshots will be automatically created using zstd. Do you have suggestions on making the text clearer?

Of course, users can manually recompress the snapshots, but I don't think it's worth it to explain that - and it's a way users could mess things up too!

[deviantintegral]

I've added version checks for zstd, let's see if that fixes things.

[tyler36]

Any new snapshots will be automatically created using zstd. Do you have suggestions on making the text clearer?

After thinking about it for a bit, the only suggestion I could come up with was the follow. It still feels too wordy though.

This snapshot was created using gzip. For improved performance, consider creating a new snapshot which now uses zstd.

[rfay]

I don't think this needs to be mentioned. gzip will be old very soon.

[rfay]

(Rebased)
Thanks for the great contribution! My experience with my old hugedb (289MB sql.gz, snapshot 929M gzipped, 799M zst):
6:30 to import from sql.gz
0:26 to snapshot with zstd, 3:44 to snapshot with gzip (HEAD)
1:30 to snapshot restore with zstd, 1:55 to restore with gzip

[stasadev]

Looks good to me, thank you!

---

MySQL 8.0, 283 MB sql (51 MB sql.gz)
v1.24.10: snapshot 38s, restore 47s
this PR: snapshot 9s, restore 43s

---

I added back support for mariadb:5.5 gz snapshots, and fixed postgres:9.

---

It makes me wonder, "How do I change to zstd?".

Any new snapshots will be automatically created using zstd. Do you have suggestions on making the text clearer?

I rephrased this warning to:

-This snapshot was created with gzip. It's recommended to create a new snapshot using zstd for better performance.
+This snapshot uses gzip compression. Creating a new snapshot will automatically use faster zstd compression.