Skip to content

test: document compressed batch delta interop upstream limitation#3231

Merged
oferchen merged 2 commits into
masterfrom
worktree-fix-compressed-batch-interop
Apr 17, 2026
Merged

test: document compressed batch delta interop upstream limitation#3231
oferchen merged 2 commits into
masterfrom
worktree-fix-compressed-batch-interop

Conversation

@oferchen
Copy link
Copy Markdown
Owner

Summary

  • Add test_upstream_compressed_batch_self_roundtrip interop test that verifies upstream rsync can read its own compressed delta batch files (using --compress-choice=zlib workaround), and that oc-rsync can also read them
  • Document the upstream rsync 3.4.1 limitation where the batch format does not record which compression algorithm was used - read-batch always assumes CPRES_ZLIB (compat.c:194-195), but write-batch may auto-negotiate zstd on builds with SUPPORT_ZSTD
  • Add documentation across batch_support.rs, replay.rs, compressed_writer.rs, flags.rs, and shell test scripts explaining the design decision and upstream limitation
  • Add unit test verifying do_compression is always false in oc-rsync batch context regardless of compression config

Background

Task #1705 investigated "WARNING: failed verification" errors when reading compressed batch files with delta transfers. The investigation found this is an upstream rsync 3.4.1 limitation, not an oc-rsync bug:

  1. Upstream tees raw compressed wire bytes to the batch file and sets do_compression=true in stream flags
  2. The batch format does not record which compression algorithm was used
  3. On read-batch, parse_compress_choice() maps do_compression=1 to CPRES_ZLIB regardless of the actual algorithm
  4. If the original transfer auto-negotiated zstd, the batch file contains zstd data but the reader uses a zlib decoder

oc-rsync avoids this entirely by recording uncompressed data in batch files (do_compression=false), making batch files portable across all compression backends.

Test plan

  • CI passes: fmt+clippy, nextest, Windows, macOS, Linux musl
  • New interop test upstream-compressed-batch-self-roundtrip passes
  • Existing compressed-batch-delta-interop test continues to pass
  • New unit test batch_context_do_compression_false_regardless_of_compress_config passes

@github-actions github-actions Bot added the test label Apr 17, 2026
Add upstream self-roundtrip test for compressed batch delta transfers
to document the upstream rsync 3.4.1 limitation where the batch format
does not record which compression algorithm was used. Upstream's
read-batch always forces CPRES_ZLIB (compat.c:194-195), but
write-batch may auto-negotiate zstd, producing unreadable batch files.

oc-rsync avoids this by recording uncompressed data in batch files
(do_compression=false in stream flags). Document this design decision
across batch_support.rs, replay.rs, compressed_writer.rs, and flags.rs.

Refs: #1705
Upstream rsync 3.4.1 cannot read back its own compressed delta batch
files - the batch writer records raw compressed tokens but the reader
expects uncompressed data. This is an upstream limitation, not an
oc-rsync bug.
@oferchen oferchen force-pushed the worktree-fix-compressed-batch-interop branch from 4c97534 to e5a2c16 Compare April 17, 2026 18:43
@oferchen oferchen merged commit 151881b into master Apr 17, 2026
43 checks passed
@oferchen oferchen deleted the worktree-fix-compressed-batch-interop branch April 17, 2026 19:16
oferchen added a commit that referenced this pull request May 1, 2026
…interop

test: document compressed batch delta interop upstream limitation
oferchen added a commit that referenced this pull request May 5, 2026
…interop

test: document compressed batch delta interop upstream limitation
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant