rsync over ssh with relative basis --link-dest=../snap.1 can fail with "failed verification -- update discarded" in 3.4.3

[JetAppsClark]

Title

--link-dest=../snap.1 over SSH can fail with failed verification -- update discarded in rsync 3.4.3

Summary

I am seeing reproducible failed verification -- update discarded errors with rsync 3.4.3 when using --link-dest over SSH with a relative basis path.

The reproduction stages/generated files into a temporary source directory, syncs them to a remote snapshot directory, then creates a second snapshot using --link-dest=../snap.1.

The first baseline snapshot succeeds, but the second snapshot using the relative --link-dest basis fails with exit code 23 and many failed verification -- update discarded errors.

A later comparison using an absolute --link-dest path to an existing snapshot basis succeeds.

This appears possibly related to the recent failed verification -- update discarded reports, but this reproduction does not require symlinked receiver directories.

Possibly related issues:

• rsync: failed verification -- update discarded - regression from CVE fixes #715
• rsync: failed verification -- update retained - no symlinks regression from CVE fixes #724
• Security fix breaks --link-dest via rsync daemon #915

NOTE: This also affects backported rsync versions when they're used on the Receiver:

Debian: 3.4.1+ds1-5+deb13u3 / 3.2.7-1+deb12u5 / 3.2.3-4+deb11u3
Ubuntu: 3.2.7-1ubuntu1.4

Environment

Sender/client:

rsync version 3.4.3 protocol version 32
rsync path: /usr/local/jetapps/usr/bin/rsync

Transport:

rsync over SSH

Remote receiver:

root@remote-host

SSH command used by rsync:

ssh -p 22 -o BatchMode=yes -o StrictHostKeyChecking=accept-new -i /path/to/test_key

Reproduction script

I used the attached script:

rsync-failed-verification-repro.sh

The script does the following:

1. Creates a temporary local staging directory.
2. Generates config-like files into the staging directory.
3. Rsyncs the staged files to a remote snap.1 directory.
4. Regenerates/copies identical staged files.
5. Rsyncs the staged files to remote snap.2 using:

--link-dest=../snap.1

6. Continues with additional comparison tests:
   • snap.5a using a relative --link-dest
   • snap.5b using an absolute --link-dest

Command used

RSYNC_BIN=/usr/local/jetapps/usr/bin/rsync \
SSH_KEY=/path/to/test_key \
REMOTE=root@remote-host \
bash rsync-failed-verification-3.4.3.sh

Actual result

snap.1 succeeds:

Creating snap.1 baseline...
snap.1 return code: 0
snap.1 log: /tmp/rsync_724_remote_repro.EDbZrr/rsync_snap1_baseline.log

snap.2 fails when using the relative --link-dest=../snap.1 basis:

Creating snap.2 baseline using snap.1...
snap.2 return code: 23
snap.2 log: /tmp/rsync_724_remote_repro.EDbZrr/rsync_snap2_baseline.log

WARNING: snap.2 baseline already reproduced the rsync verification failure.
Continuing so the relative and absolute snap.5 tests can still run.

The log contains many entries like:

ERROR: bandwidth.tally failed verification -- update discarded.
ERROR: bandwidth.tally.cache failed verification -- update discarded.
ERROR: crontab.conf failed verification -- update discarded.
ERROR: domains.list failed verification -- update discarded.
ERROR: du_breakdown.list failed verification -- update discarded.
ERROR: httpd.conf failed verification -- update discarded.
ERROR: ip.list failed verification -- update discarded.
ERROR: user.conf failed verification -- update discarded.
ERROR: user.db failed verification -- update discarded.
ERROR: dns/zone-001.db failed verification -- update discarded.
ERROR: dns/zone-002.db failed verification -- update discarded.

The failing rsync operation is equivalent to:

/usr/local/jetapps/usr/bin/rsync -aH --delete \
  -e "ssh -p 22 -o BatchMode=yes -o StrictHostKeyChecking=accept-new -i /path/to/test_key" \
  --link-dest=../snap.1 \
  /tmp/rsync_724_remote_repro.EDbZrr/staging/ \
  root@remote-host:/tmp/rsync_724_remote_repro.EDbZrr/rsync-sim/config/snap.2/

Expected result

The snap.2 transfer should complete successfully, or at least behave the same as when using an absolute --link-dest path to the same basis directory.

The staged source files are generated/copied into the source staging directory before each sync and should be identical between the snapshots.

Comparison result

After the snap.2 failure, the script continued and ran two additional tests.

Relative --link-dest to snap.2:

Running:
  /usr/local/jetapps/usr/bin/rsync -aH --delete -e "ssh -p 22 -o BatchMode=yes -o StrictHostKeyChecking=accept-new -i /path/to/test_key" --link-dest=../../config/snap.2 /tmp/rsync_724_remote_repro.EDbZrr/staging/ root@remote-host:/tmp/rsync_724_remote_repro.EDbZrr/rsync-sim/config/snap.5a/

Return code: 0
Log: /tmp/rsync_724_remote_repro.EDbZrr/rsync_snap5a_relative.log

Absolute --link-dest to snap.2:

Running:
  /usr/local/jetapps/usr/bin/rsync -aH --delete -e "ssh -p 22 -o BatchMode=yes -o StrictHostKeyChecking=accept-new -i /path/to/test_key" --link-dest=/tmp/rsync_724_remote_repro.EDbZrr/rsync-sim/config/snap.2 /tmp/rsync_724_remote_repro.EDbZrr/staging/ root@remote-host:/tmp/rsync_724_remote_repro.EDbZrr/rsync-sim/config/snap.5b/

Return code: 0
Log: /tmp/rsync_724_remote_repro.EDbZrr/rsync_snap5b_absolute.log

Because the snap.5a relative test returned 0, I am not claiming that every relative --link-dest path fails. The consistent failure in this reproduction is the earlier snap.2 creation using:

--link-dest=../snap.1

Notes / questions

Is --link-dest=../snap.1 expected to be resolved relative to the destination directory being created on the receiver side?

In this reproduction, the destination is /tmp/rsync_724_remote_repro.EDbZrr/rsync-sim/config/snap.2/ so I would expect ../snap.1 to resolve to /tmp/rsync_724_remote_repro.EDbZrr/rsync-sim/config/snap.1/

If this understanding is incorrect, please let me know. Otherwise, this appears to be a regression in relative --link-dest basis handling during rsync-over-SSH transfers in 3.4.3.

Attachments

rsync-failed-verification-repro.sh