Skip to content

sync: stop re-reading a file the scan cache already settled - #59

Merged
myobie merged 1 commit into
mainfrom
sync/materialize-uses-scan-cache
Aug 19, 2026
Merged

sync: stop re-reading a file the scan cache already settled#59
myobie merged 1 commit into
mainfrom
sync/materialize-uses-scan-cache

Conversation

@myobie

@myobie myobie commented Aug 19, 2026

Copy link
Copy Markdown
Collaborator

Closes #56.

The fault

materialize_tracked read and hashed EVERY present file on EVERY pass. On a
converged tree that answer is always "unchanged", so the read and the hash were
pure waste.

Measured on the live Silber daemon, version 0.2.0+fff659e, before #57 landed:
one entry re-read and re-hashed 70,157,702 bytes every 0.51 s, which is
136 MB/s. blake3_hash_many_neon was 14.22 percent of one core, the largest
single consumer. 99.6 percent of those bytes belonged to three files that never
change.

Read this after #58, because it changes what this PR is worth

#58 fixed the cause: a converged tree no longer full-scans twice a second. At
the expected one pass per 300 s, this entry's re-hash cost falls from 136 MB/s
to about 0.23 MB/s on its own.

So this is no longer the fix. It is defense in depth, and it still earns its
place for two reasons:

I would not have merged this before #58, and cos sequenced it that way.

The change

This asks the same question scan_folder asks, from the same evidence: size and
both mtime components byte-identical to what THIS MACHINE last observed.

Trusting that is the existing contract, not a new one. scan_folder already
runs first in every pass and already trusts the cache, so re-deriving the
opposite answer here, from the same disk, was incoherent rather than safe.
materialization_does_not_manufacture_an_mtime_collision already pins that a
local cache entry is trusted by design, and that only this machine can create
one, so no peer can manufacture a hit.

Anything unknown, differing, or unreadable takes the full read-and-hash path
exactly as before.

The behaviour change I am not hiding

Drift that preserves both size and mtime is no longer repaired by
materialization. scan_folder would not have noticed it either, so that repair
was never coherent. That is the argument for the change, not a caveat I found
afterwards.

Was the bypass deliberate?

cos asked me to check this before writing anything, and to stop if the bypass
was load bearing. It is not.

There is no comment, no test and no rationale for materialize re-reading. The
documented incident in that code, Linux CI run 30814462024 attempt 4 with 24
republishes from v44 to v67, was the SCAN trusting a REPLICATED manifest. PR 47
fixed that by keying the cache locally, and its test states the resulting
position outright: "a local cache entry is trusted by design; only its ORIGIN
changed".

Tests

A read cannot be observed directly, so the test makes the read CHANGE THE
ANSWER. The disk carries bytes that differ from the recorded hash while wearing
the same size and mtime. A materialization that reads sees drift and republishes
it; one that trusts the cache leaves it alone.

Run against the pre-fix code it fails, on the republish assertion.

A control asserts a file the cache does NOT cover is still read and hashed, so
the fast path cannot quietly become "never check anything". The control passes
both before and after the fix, so it is a control and not a mirror.

Verification

macOS on the rebased branch: 231 lib, 3 sync_slice, 21 local_slice. engine.rs
is rustfmt clean and clippy matches main with an empty set difference both
ways.

I also tested this together with #58 on a throwaway branch before either merged,
because a clean merge-tree says nothing about behaviour and this fast path
depends on the cache freshness that #58 makes rare. It holds: sync_once always
scans immediately before materializing under one operation guard, and the
standalone materialize_entry path is stat-validated, so staleness
self-corrects.

That is a macOS claim. This PR's CI run is the Linux face and I read the job
steps before calling it green.

Issue 56. materialize_tracked read and hashed EVERY present file on EVERY
pass. On a converged tree that answer is always "unchanged", so the read and
the hash were pure waste.

Measured on the live Silber daemon, version 0.2.0+fff659e. One entry re-read
and re-hashed 70,157,702 bytes every 0.51 seconds, which is 136 MB/s.
blake3_hash_many_neon was 14.22 percent of one core, the largest single
consumer. 99.6 percent of those bytes were three files that never change.

This asks the same question scan_folder asks, from the same evidence: size and
both mtime components byte-identical to what THIS MACHINE last observed.

Trusting that is the existing contract, not a new one. scan_folder already runs
first in every pass and already trusts the cache, so re-deriving the opposite
answer here, from the same disk, was incoherent rather than safe.
materialization_does_not_manufacture_an_mtime_collision already pins that a
local cache entry is trusted by design, and that only this machine can create
one, so no peer can manufacture a hit.

Anything unknown, differing, or unreadable takes the full read-and-hash path
exactly as before.

The new test makes the read CHANGE THE ANSWER, because a read cannot be
observed directly. The disk carries bytes that differ from the recorded hash
while wearing the same size and mtime. A materialization that reads sees drift
and republishes it; one that trusts the cache leaves it alone. I ran that test
against the pre-fix code and it fails, on the republish assertion.

A control asserts a file the cache does NOT cover is still read and hashed, so
the fast path cannot quietly become "never check anything". The control passes
both before and after the fix, so it is a control and not a mirror.

This does not change the number of passes. The declarations entry still walks
14,368 files about twice a second, which is roughly 8 percent of a core in
getdirentries and glob matching. That is issue 57 and it is separate.
@myobie
myobie merged commit 09b8df4 into main Aug 19, 2026
2 checks passed
@myobie
myobie deleted the sync/materialize-uses-scan-cache branch August 19, 2026 05:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

materialize_tracked re-reads and re-hashes every present file on every pass, ignoring the scan cache

1 participant