fix: defer metadata application until after disk thread commit#2344
Merged
fix: defer metadata application until after disk thread commit#2344
Conversation
The decoupled receiver was applying file metadata (mtime, permissions, ownership) immediately after queueing the commit to the disk thread, before the disk thread had actually flushed and renamed the temp file to its final path. This caused: 1. mtime not preserved in daemon mode (utimensat on non-existent path) 2. Incremental sync 6.5x slower than upstream (quick-check always failed because destination mtimes never matched source) Fix: collect deferred metadata entries during the transfer loop and apply them after drain_all_results() confirms all files are committed. Mirrors upstream finish_transfer() → set_file_attrs() which runs after do_rename(fnametmp, fname) in receiver.c. Benchmark (10K files, 422MB, daemon mode): - Incremental: 923ms → 121ms (1.13x faster than upstream's 137ms) - Initial copy: no regression (627ms, 1.65x faster than upstream)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Root Cause
The decoupled receiver architecture (introduced in PR #2338) separated network I/O from disk I/O via a bounded channel + disk commit thread. However, metadata was applied on the network thread immediately after queueing the commit, before the disk thread had flushed and renamed the temp file.
Sequence (buggy):
Commitmessage to disk threadFix
Defer metadata application to after
drain_all_results()confirms all files are committed. Mirrors upstreamfinish_transfer()→set_file_attrs()which runs afterdo_rename(fnametmp, fname)inreceiver.c.Benchmark (10K files, 422MB, daemon mode)
Test plan